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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  (ARI)  supports  the  Army  with  research  and 
development  on  manpower,  personnel,  training,  and  human  perfor¬ 
mance  issues.  One  concern  that  underlies  all  of  these  issues  is 
the  mental  workload  imposed  on  and  experienced  by  the  operators 
of  newly  emerging,  high-technology  systems  and  the  impact  of  that 
workload  on  operator  and  system  performance.  The  Fort  Bliss 
Field  Unit  is  conducting  exploratory  development  research  for 
establishment  of  an  operator  workload  assessment  program  for  the 
U.S.  Army . 


This  technical  report  documents  the  ninth  field  study  of 
operator  workload  conducted  by  the  Fort  Bliss  Field  Unit.  The 
Stingray  system,  which  is  the  focus  of  the  present  study,  is  the 
fifth  system  studied  in  the  Fort  Bliss  workload  research  program. 
This  particular  study  was  conducted  at  the  request  of  the  Air 
Defense  Artillery  Test  Directorate  of  the  U.S.  Army  Test  and 
Experimentation  Command.  The  results  of  the  field  study  contrib¬ 
uted  significantly  to  key  goals  and  objectives  of  the  system 
proponents,  the  system  developers,  and  the  testing  agency.  Fur¬ 
ther,  the  workload  data  and  other  information  obtained  in  the 
study  produced  several  unexpected  findings.  For  example,  system 
operators  changed  a  number  of  standard  operating  procedures  in 
response  to  the  workload  they  experienced.  Findings  such  as 
these  have  important  implications  for  the  continuing  development 
and  deployment  of  the  Stingray  system. 

In  addition,  the  present  study  makes  important  contributions 
to  the  technology  base.  The  results  of  this  effort  have  added 
substantially  to  the  knowledge  base  for  a  number  of  important 
methodological  issues  in  the  areas  of  workload  assessment  and 
management  and  to  the  U.S.  Army's  understanding  of  the  impact  of 
operator  workload  on  system  performance  and  force  effectiveness. 
Perhaps  most  important,  the  results  of  this  study  confirm  that 
the  experiences  reported  by  a  system  operator  are  both  reliable 
and  unique  sources  of  information  for  system  developers. 


Technical  Director 
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ASSESSMENT  OF  WORKLOAD  IN  A  FIELD  ENVIRONMENT:  IMPLICATIONS  FOR 
SOME  UNRESOLVED  WORKLOAD  ISSUES 


EXECUTIVE  SUMMARY 


Requirement: 

This  report  describes  a  study  that  evaluated  the  impact  on 
crew  workload  of  adding  the  Stingray  system  to  the  Bradley  Fight¬ 
ing  Vehicle  (BFV) .  It  offers  insight  into  the  assessment  and 
interpretation  of  operator  workload  in  field  environments. 


Procedure : 

Four  BFV  crews  participated  in  force-on-force  offensive  and 
defensive  missions  in  a  baseline  BFV  (without  Stingray)  and  a  BFV 
with  Stingray.  Workload  was  assessed  through  operator  ratings, 
post-mission  debriefs,  video  and  audio  recordings,  and  an  end-of- 
test  questionnaire.  System  performance  was  assessed  using  the 
force  effectiveness  ratio. 


Findings: 

Results  indicated  that  workload  is  dependent  on  mission  type 
(offensive  or  defensive)  and  mode  of  operation  (baseline, 
Scingray-automatic,  Stingray-semiautomatic,  or  Stingray-manual) . 
Crew  strategies  to  manage  workload  included  reallocation  of  some 
of  the  Bradley  commander  tasks  to  the  BFV  gunner  and  driver.  A 
significant  negative  relationship  was  established  between  work¬ 
load  and  force  effectiveness  (i.e.,  increases  in  crew  workload 
were  associated  with  decreases  in  force  effectiveness) .  The 
utility  of  the  workload  measurement  techniques  employed  in  the 
study  is  discussed  in  terms  of  the  practical  and  procedural 
significance  of  the  study  results. 


Utilization  of  Findings: 

The  findings  of  this  Stingray  workload  assessment  are 
included  in  the  Concept  Evaluation  Program  (CEP)  final  test 
report.  Based  on  information  gained  through  the  Stingray 
workload  assessment,  important  revisions  were  incorporated  into 
the  Stingray  System  MANPRINT  Management  Plan.  Two  conference 
papers  based  on  this  study  were  published.  "Assessment  of  Work¬ 
load  in  a  Field  Environment"  was  published  in  the  Proceedings  of 


vii 


the  34th  Annual  Meeting  of  the  Human  Factors  Society,  and  "Gen¬ 
eral  Issues  of  Operator  Workload"  was  published  in  the  Proceed¬ 
ings  of  the  12th  Biennial  Psychology  in  the  Department  of  Defense 
Symposium.  This  Stingray  workload  evaluation  study  serves  as  a 
model  for  Army  programs  that  must  address  operator  workload 
issues  and  concerns. 
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ASSESSMENT  OF  WORKLOAD  IN  A  FIELD  ENVIRONMENT: 
IMPLICATIONS  FOR  SOME  UNRESOLVED  WORKLOAD  ISSUES 


INTRODUCTION 


Purpose  of  This  Report 

The  research  effort  described  and  discussed  in  this  report 
had  two  major  objectives:  (a)  to  provide  information  of 
immediate  practical  value  to  the  developers  and  proponents  of  an 
emerging  weapon  system,  and  (b)  to  expand  and  enhance  the  Army's 
capability  to  assess  and  understand  the  impact  of  operator 
workload. 

This  introductory  section  is  used  to  provide  background 
information  concerning  the  problem  created  when  new  Army  systems 
impose  increased  demands  on  soldiers  for  mental  skills,  the 
concept  and  measurement  of  operator  workload,  and  the  operator 
workload  research  program  of  the  Fort  Bliss  Field  Unit  of  the 
U.S.  Army  Research  Institute  (ARI) .  This  section  also  contains  a 
description  of  an  operational  field  test  of  the  Stingray  system, 
a  brief  discussion  of  several  key  operator  workload  issues,  and  a 
list  of  specific  research  questions  this  effort  was  to  address. 

The  methods  employed  during  the  research  effort  and  the 
quantifiable  results  obtained  are  described  in  two  subsequent 
sections.  An  expanded  discussion  of  the  results  and  their 
implications  for  both  the  specific  weapon  system  under  study  and 
the  more  general  operator  workload  issues  are  contained  in  a 
separate  section  of  the  report.  The  conclusions  that  can  be 
drawn  from  this  research  effort  are  in  the  final  section  of  the 
report;  it  is  argued  that  the  results  of  this  research  have 
important  implications  for  future  operator  workload  assessment 
and  research  programs. 


Background 

Problem.  Projected  manpower  declines  coupled  with  increases 
in  personnel  costs  and  battlefield  sophistication  has  prompted  an 
increased  reliance  on  high  technology  equipment  in  new  Army 
systems.  As  technology  has  changed,  so  has  the  role  of  the 
operator  of  this  equipment.  Tasks  assigned  to  the  operator  have 
shifted  from  those  that  primarily  require  physical  exertion  and 
motor  coordination  to  those  that  increasingly  impose  perceptual 
and  cognitive  demands. 
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While  technological  advancements  are  designed  to  increase 
system  capability,  it  is  critical  to  ensure  that  they  do  not  also 
cause  an  unanticipated  increase  in  the  requirement  for  operator 
mental  skills.  Task  demands  greater  than  an  operator's  capacity 
to  respond  may  result  in  undesirable  consequences,  such  as  a 
degradation  in  mission  performance  or  a  compromise  in  system 
safety. 

Definition  and  assessment  of  operator  workload.  The  concept 
of  operator  workload  is  generally  defined  in  terms  of  an 
interaction  between  the  demands  imposed  upon  the  operator  and  the 
capabilities  of  the  operator  to  successfully  respond  to  those 
demands.  The  conceptual  foundations  of  operator  workload,  to 
include  the  relationship  between  workload  and  performance,  have 
been  discussed  extensively  in  numerous  recent  publications  (e.g., 
Gopher  &  Donchin,  1986;  Lysaght  et  al.,  1989).  Likewise,  there 
have  been  numerous  reviews  of  techniques  available  to  assess 
operator  workload  (e.g.,  Lysaght  et  al.,  1989;  O'Donnell  & 
Eggemeier,  1986;  Wierwille  &  Williges,  1980). 

While  there  has  been  considerable  attention  given  to 
operator  workload  and  its  assessment,  most  operator  workload 
research  has  occurred  in  academic  laboratory  settings.  The 
limited  applied  research  has  been  principally  associated  with  the 
operation  of  aviation  systems.  Consequently,  there  remain  many 
questions  concerning  the  application  of  operator  workload  to  the 
design  and  development  of  new  systems.  In  addition,  there  is  a 
void  in  specific  guidance  concerning  how  best  to  conduct  an 
operator  workload  assessment  and  analysis  program  for  a  given 
system. 

The  ARI  operator  workload  (OWL)  program.  In  response  to  the 
need  for  useful  guidance  in  the  assessment  of  operator  workload, 
the  ARI  Field  Unit  at  Fort  Bliss,  Texas,  launched  a  multi-year 
research  effort  called  the  Operator  Workload  (OWL)  Program.  The 
researchers  who  conducted  this  program  defined  the  needs  of  the 
Army  (Hill  et  al.,  1987),  critically  reviewed  and  evaluated  the 
operator  workload  concept  and  workload  assessment  techniques 
(Lysaght  et  al.,  1989),  and  conducted  field  assessments  of 
operator  workload  for  the  following  Army  systems: 

•  Aquila  remotely  piloted  vehicle  (RPV)  (Byers,  Bittner, 
Hill,  Zaklad,  &  Christ,  1988), 

•  Line-of-sight  forward  heavy  (LOS-F-H)  component  of  the 
Forward  Area  Air  Defense  System  (FAADS)  (Bittner,  Byers, 
Hill,  Zaklad,  &  Christ,  1989;  Hill,  Byers,  Zaklad, 
Bittner,  &  Christ,  1988;  Hill,  Byers,  Zaklad,  &  Christ, 
1989a,  1989b;  Hill,  Zaklad,  Bittner,  Byers,  &  Christ, 
1988) ,  and 
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•  UH-60  Black  Hawk  helicopter  (Iavecchia,  Linton,  Bittner, 

&  Byers,  1989) . 

The  results  of  these  efforts  in  the  ARI  OWL  program  were  the 
basis  for  the  development  and  publication  of  two  research 
products: 

•  A  pamphlet  for  Army  managers  describes  the  need  and  some 
procedures  for  incorporating  operator  workload  issues  and 
concerns  into  the  Army  materiel  acquisition  process 
(Christ,  Bulger,  Hill,  &  Zaklad,  1990),  and 

•  A  computer-based  tool,  the  operator  workload  knowledge- 
based  expert  system  tool  (OWLKNEST) ,  and  its  user's 
manual,  the  Handbook  for  Operating  the  OWLKNEST 
Technology  (HOOT) ,  provide  guidance  for  selecting  the 
workload  assessment  techniques  most  appropriate  for  any 
specific  situation  (Harris,  Hill,  Lysaght,  &  Christ, 

1991) . 

The  results  of  the  OWL  Program  were  used  to  identify  gaps  in 
our  knowledge  and  understanding  of  operator  workload  concepts  and 
assessment  techniques  (Christ,  Zaklad,  Bittner,  Hill,  &  Linton, 
1989;  Sams  &  Christ,  1990).  Additional  research  is  required  to 
fill  these  knowledge  gaps. 


The  Stingray  Concept  Evaluation  Program  (CEP) 

In  September,  1989,  the  Air  Defense  Artillery  Board  of  the 
U.S.  Army  Test  and  Experimentation  Command  (TEXCOM)  formally 
requested  that  the  Fort  Bliss  Field  Unit  of  ARI  support  an 
operational  field  test  of  a  proposed  improvement  to  the  Bradley 
fighting  vehicle  (BFV) .  This  product  improvement  consisted  of 
adding  a  new  weapon  system  called  Stingray  to  the  weapon  system 
ensemble  al  eady  available  to  the  BFV  crew.  The  field  test  was 
the  basis  for  a  concept  evaluation  program  (CEP)  for  the  t  tingray 
system.  The  major  objectives  of  the  field  test  were  to  (a) 
evaluate  the  impact  on  force  effectiveness  of  adding  Stingray  to 
one  of  four  BFVs  in  a  mechanized  infantry  platoon  and  (b) 
determine  any  impact  of  Stingray  operations  on  the  workload  of 
the  Bradley  commander  (BC) . 

The  invitation  to  participate  in  the  field  test  of  the 
Stingray  system  presented  an  opportunity  to  satisfy  two  major 
missions  of  the  ARI  field  unit.  It  permitted  the  field  unit  to 
fulfill  its  mission  to  generally  support  U.S.  Army  initiatives; 
in  this  case,  to  address  the  major  objectives  of  the  Stingray 
CEP.  It  also  permitted  the  field  unit  to  pursue  its  mission  to 
conduct  behavioral  science  research;  in  this  case,  to  continue 
the  research  effort  begun  under  the  ARI  OWL  research  program. 
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Brief  description  of  the  Stingray  system.  Stingray  is  a 
laser  system  designed  to  augment  the  more  conventional  weapons  of 
mechanized  infantry  units.  The  Stingray  system  will  detect  and 
temporarily  deprive  the  opposing  force  of  the  use  of  their 
optical  and  electro-optical  devices.  Therefore,  the  Stingray 
system  is  used  principally  as  a  countermeasure  system. 

The  Stingray  system  has  three  modes  of  operation,  each 
requiring  different  types  and  amounts  of  action  from  the  BC. 

Once  the  Stingray  system  is  activated  in  the  automatic  mode, 
there  are  no  additional  actions  required  of  the  BC;  the  system 
will  search  for,  detect,  and  countermeasure  optical  and  electro- 
optical  capabilities  of  the  opposing  force.  In  the  semiautomatic 
mode,  Stingray  automatically  scans  for  targets  and,  if  any  are 
detected,  presents  them  for  10  seconds  on  a  visual  display  unit 
( VDU)  inside  the  BFV  turret  for  evaluation  by  the  BC.  If  the  BC 
decides  that  the  target  is  a  threat,  the  BC  countermeasures  it  by 
centering  the  VDU  cross  hairs  on  the  target  and  depressing  a 
trigger  switch  on  the  turret  joystick.  The  manual  mode  of 
Stingray  requires  the  BC  to  control  the  search  pattern  with  the 
joystick,  detect  and  evaluate  targets  shown  in  the  visual 
display,  and  countermeasure  threats  with  a  trigger  pull. 

For  any  of  its  three  modes  of  operation.  Stingray  signals  a 
target  detection  with  an  auditory  alarm  either  through  speakers 
in  the  turret  or  through  a  headset  incorporated  into  the  Combat 
Vehicle  Crewmember  (CVC)  helmet.  Once  Stingray  is  used  to 
countermeasure  an  optical  or  electro-optical  capability  of  the 
opposing  force,  target  position  information  (i.e.,  range  and 
azimuth)  is  displayed  to  both  the  BC  and  gunner  as  an  aid  for 
target  engagement  with  conventional  BFV  weapons  (i.e.,  the  TOW 
missile  launcher  or  the  25-mm  gun) . 


Operator  Workload  Research  Issues  and  the  Opportunities  Presented  in  the  Stingray 
CEP 


Participation  in  the  Stingray  CEP  permitted  research  to  be 
conducted  that  addresses  several  unresolved  issues  in  the 
assessment  and  interpretation  of  operator  workload.  These  issues 
are  briefly  described  in  the  succeeding  paragraphs. 

"CreW1  workload.  A  key  CEP  test  objective  was  to  determine 
any  impact  of  Stingray  operations  on  the  workload  of  a  single 
member  of  the  BFV  crew,  the  BC.  However,  after  studying  the 
functions  of  Stingray  and  the  required  tasks  of  the  Bradley  crew, 
it  was  concluded  that  the  workload  of  the  entire  crew  could  and 
should  be  evaluated.  This  expansion  of  the  test  objective  is  an 
important  change  from  most  previous  operator  workload  studies. 
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In  a  typical  study,  the  impact  of  workload  is  examined  for  only  a 
single  system  operator.  However,  as  noted  by  Hart  (1989b) ,  "The 
effects  of  crew  coordination  and  shared  task  performance  on 
workload  are  virtually  unknown.  They  are  rarely  studied  in  the 
laboratory,  and  although  they  certainly  affect  the  workload  of 
individual  crew  members  in  operational  situations,  they  have  not 
been  studied  systematically  there  either"  (p.22). 

The  Stingray  CEP  permitted  detailed  workload  analyses  to  be 
performed  for  the  BC  and  gunner,  and  a  more  cursory  analysis  to 
be  performed  for  the  driver.  The  effects  of  workload  on  the 
entire  crew  could  therefore  be  examined  under  various  conditions 
in  a  baseline  BFV  (i.e.,  a  BFV  without  Stingray  capabilities)  or 
in  the  Stingray  BFV  in  each  of  its  three  modes  of  operation. 

Workload  management  strategies.  Participation  in  the 
Stingray  CEP  provided  an  opportunity  to  identify  and  examine 
consequences  of  any  workload  management  strategies  employed  by 
BFV  crew  members.  Operator  strategies  play  an  important  role 
when  examining  the  relationship  among  objective  task  demands, 
experienced  workload,  and  system  performance.  However,  previous 
research  has  virtually  ignored  the  role  of  this  important 
variable  (Hart,  1989a) . 

When  faced  with  high  task  demands,  operators  may  adopt  any 
number  of  strategies  in  an  attempt  to  maintain  an  acceptable 
level  of  performance.  For  individual  operators,  required  tasks 
may  be  rescheduled,  and  tasks  which  are  not  absolutely  required 
may  be  shed  and  simply  not  performed.  For  crew  operations,  as 
occur  in  the  BFV,  tasks  may  be  reallocated  among  crew  members  in 
an  effort  to  redistribute  the  workload.  These  dynamic  responses 
of  system  operators  have  important  implications  for  new  systems 
in  terms  of  equipment  design,  task  allocation,  and  training. 

Workload  and  performance.  There  is  much  debate  among 
researchers  about  the  "validity"  of  operator  workload  assessment 
techniques  as  measured  by  the  relationship  between  operator 
workload  and  performance  (Sams  &  Christ,  1990) .  For  example, 

Hart  (1989b)  contends  that  there  is  a  difference  between 
measuring  task  performance  and  measuring  the  amount  of  effort 
expended  to  perform  the  task.  Operators  may  report  high  workload 
(e.g.,  by  means  of  rating  scales)  and  still  be  able  to  exert  the 
extra  effort  required  to  achieve  desired  performance  levels. 
However,  under  some  conditions,  such  as  after  a  period  of 
sustained  operations,  the  operator  may  not  be  able  to  call  up 
additional  effort  and  performance  may  be  degraded. 

In  contrast,  other  researchers  posit  that  objective 
performance  measures  for  the  operator  and  system  are  more 
accurate  indicators  of  the  effects  of  workload  than  the 
subjective  experience  of  the  operator  (Seven,  1989) .  These 
measures  might  include  crew  performance  time  and  error,  as  well 
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as  the  resulting  system  performance.  Wickens  (1989)  states  that 
the  "ultimate  design  criterion  should  be  satisfactory  performance 
rather  than  adequate  subjective  opinion"  (p.  271) . 

In  the  Stingray  CEP,  both  "subjective"  measures  of  workload 
(i.e.,  operator  ratings)  and  "objective"  measures  (i.e.,  crew  and 
system  performance  measures)  were  to  be  obtained.  The 
association  between  these  two  types  of  measures  could  then  be 
analyzed.  Aside  from  the  theoretical  research  interest,  a 
demonstrated  relationship  between  the  workload  experienced  by  the 
crew  members  and  the  performance  of  the  system  would  be  of 
significant  value  to  the  Stingray  operational  test  findings. 

Prospective  and  retrospective  rating  techniques.  Two  issues 
associated  with  using  operator  workload  ratings  relate  to  the 
timing  of  the  measurements.  On  the  one  hand,  rating  techniques 
can  be  used  "prospectively"  to  predict  the  amount  of  workload 
that  will  be  experienced  by  the  operator  of  either  a  future 
system  or  an  existing  system  that  is  to  be  modified  or  operated 
in  some  innovative  manner.  On  the  other  hand,  rating  techniques 
can  be  used  "retrospectively"  to  report  the  amount  of  workload 
that  was  experienced  by  the  operator  of  a  system  during  an 
earlier  period  of  time.  Previous  studies  in  the  ARI  OWL  program 
examined  both  of  these  issues  but  were  unable  to  firmly  establish 
the  validity  of  either  the  prospective  or  retrospective  rating 
techniques  used. 

Hill,  Byers,  Zaklad,  Bittner,  and  Christ  (1988)  showed  that 
operators  of  a  prototype  system  can  prospectively  rate  the 
workload  that  would  be  associated  with  system  configurations  and 
with  system  employment  procedures  they  had  not  yet  experienced. 
Bittner,  Byers,  Hill,  Zaklad,  and  Christ  (1989)  found  no 
difference  in  the  prospective  workload  ratings  of  experienced 
operators  and  subject  matter  experts  who  had  no  actual  experience 
operating  the  system  under  study.  Both  groups  were  to  rate  the 
workload  that  would  be  experienced  by  system  operators  during 
proposed  but  untried  generic  mission  scenarios. 

In  most  operational  tests  of  emerging  systems,  workload 
ratings  are  obtained  retrospectively,  most  typically  within  a  few 
minutes  or  hours  after  completing  system  operations.  For 
example.  Hill,  Byers,  Zaklad,  and  Christ  (1989b)  had  operators  of 
a  mobile  air  defense  system  rate  workload  within  one  hour  after 
the  completion  of  four-hour  missions.  However,  in  other  field 
studies,  the  first  opportunity  to  assess  operator  experiences  may 
not  occur  until  after  some  longer  period  of  time  has  passed.  For 
example,  Hill,  Zaklad,  Bittner,  Byers,  and  Christ  (1988)  could 
not  gain  access  to  the  operators  of  the  same  mobile  air  defense 
system  until  ten  weeks  after  the  field  test  was  completed.  In 
both  of  these  retrospective  workload  rating  studies,  system 
operators  rated  the  workload  they  had  previously  experienced 
after  viewing  video  tapes  of  their  earlier  performance.  However, 
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neither  of  these  studies  was  able  to  establish  that  retrospective 
ratings  were  valid  indicators  of  the  workload  experienced  when 
the  task  or  mission  was  actually  being  performed. 

The  ideal  method  for  validating  either  prospective  or 
retrospective  workload  ratings  is  to  compare  them  with  real-time 
workload  ratings  obtained  while  or  immediately  after  the  operator 
performs  the  tasks  or  the  mission  of  interest.  The  Stingray  CEP 
provided  a  unique  opportunity  to  gather  workload  ratings  from  one 
set  of  subjects  prospectively  (at  the  conclusion  of  system- 
specific  training  but  prior  to  the  test  missions) ,  real-time 
(immediately  following  each  test  mission),  and  retrospectively 
(at  the  conclusion  of  all  test  missions) .  The  correspondence 
between  these  measures  will  provide  some  information  about  the 
validity  of  rating  scales  as  prospective  and  retrospective 
workload  assessment  techniques. 

Alternative  workload  assessment  techniques.  The  results 
obtained  during  the  ARI  OWL  program  have  two  important 
implications  for  how  to  successfully  measure  workload  in  field 
settings  (see  Zaklad,  Harris,  Iavecchia,  Christ,  &  Sams,  1990) . 
First  and  foremost,  the  workload  assessment  technique  must  be 
non-intrusive  since  any  extra  tasks  imposed  upon  a  system 
operator  for  the  purpose  of  assessing  workload  could  well 
interfere  with  the  operator  capability  to  perform  primary 
mission-essential  tasks.  All  experimental  variables,  conditions, 
and  procedures  which  define  the  workload  assessment  effort  must 
dovetail  with  the  objectives,  constraints,  and  schedule  of  the 
field  test.  Secondly,  since  there  are  so  many  factors  varying 
concurrently  in  a  field  setting  and  generally  insufficient 
resources  to  do  all  that  is  desirable,  it  is  necessary  to  tap 
into  multiple  sources  of  information  and  to  utilize  a  battery  of 
alternative  workload  assessment  techniques  and  procedures.  In 
these  regards,  the  actual  assessment  techniques  employed  must  be 
robust  and  must  collectively  yield  the  maximum  amount  of 
information  for  a  given  level  of  effort. 

The  techniques  available  in  the  Stingray  test  to  assess  and 
interpret  workload  included  operator  ratings  of  workload,  post¬ 
mission  debriefings  of  system  operators,  visual  and  auditory 
recordings  of  operator  and  system  performance,  and  an  end-of-test 
questionnaire.  One  research  objective  was  to  evaluate  the 
usability  and  usefulness  of  these  alternative  techniques. 

Of  particular  interest  was  the  evaluation  of  absolute  and 
relative  measurement  techniques  when  each  is  used  prospectively 
and  retrospectively  to  estimate  real-time  measures  of  workload. 
Absolute  measurement  techniques  require  an  operator  to  estimate 
the  magnitude  of  workload  associated  with  each  task  or  test 
condition  independently  of  any  other  condition.  Relative 
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measurement  techniques  require  an  operator  to  directly  compare 
the  workload  associated  with  the  different  tasks  or  conditions  of 
interest. 

The  absolute  measurement  technique  chosen  for  this  study  was 
the  Task  Load  Index  (TLX)  workload  rating  scale  (NASA-Ames 
Research  Center,  1986) .  This  particular  rating  scale  was 
selected  over  other  frequently  used  workload  rating  scales  based 
on  studies  demonstrating  its  factor  validity  and  user  acceptance, 
and  its  utility  as  both  a  prospective  and  retrospective  rating 
technique  (Byers,  Bittner,  Hill,  Zaklad,  &  Christ,  1988;  Hill, 
Zaklad,  Bittner,  Byers,  &  Christ,  1988) .  The  Subjective  Workload 
Dominance  (SWORD)  rating  scale  was  selected  as  the  relative 
measurement  technique  since  it  has  been  shown  to  be  a  sensitive 
prospective  and  retrospective  measure  of  workload  (Vidulich, 

Ward,  &  Schuren,  in  press) .  The  use  of  SWORD  in  the  present 
study  was  also  desirable  since  it  has  previously  been  contrasted 
with  TLX  (Vidulich  &  Tsang  1987,  1986),  but  has  not  been 
previously  used  in  an  Army  field  test. 

The  remaining  measurement  techniques  used  during  the 
Stingray  CEP  were  principally  selected  to  provide  information 
that  would  be  essential  for  interpreting  the  more  quantitative 
data  that  would  be  available  from  the  rating  scales.  Interviews 
and  questionnaires,  in  particular,  are  frequently  used  in  field 
test  environments  and  are  valuable  tools  for  obtaining  the 
evaluations  and  opinions  of  test  participants  (Enderwick,  1987; 
Meister,  1985,  1986). 

Specific  Workload  Research  Questions  for  the  Stingray  CEP 

The  following  questions  guided  this  research  effort: 

1.  What  impact  does  Stingray  have  on  the  workload 
experienced  by  the  BC  and  other  BFV  crew  members  as  they  perform 
mission-essential  tasks? 

2.  What  workload  management  strategies,  if  any,  are 
employed  by  the  crew  of  the  Stingray-augmented  BFV? 

3.  What  is  the  relationship  between  real-time  operator 
workload  ratings  and  both  crew  and  system  performance? 

4.  What  is  the  correlation  between  both  prospective  and 
retrospective  operator  workload  ratings  and  real-time  operator 
workload  ratings? 

5.  How  useful  are  the  various  workload  assessment 
techniques  employed  in  the  Stingray  CEP  for  assessing  and 
interpreting  operator  workload  and  performance? 


8 


METHOD 


Participants.  Force  Composition,  and  Stingray  Training 

Elements  of  the  1st  Squadron  of  the  3rd  Armored  Cavalry 
Regiment  (ACR)  served  as  players  in  the  Stingray  CEP.  A  platoon 
comprised  of  four  squads  (i.e.,  four  BFVs  and  their  crews)  served 
as  the  Stingray  test  platoon.  Other  elements  drawn  from  the  1st 
Squadron  served  as  supporting  players  of  the  Blue  force  or  as 
surrogates  of  threat  systems  of  the  Red  force.  The  Blue  force 
used  accepted  U.S.  Army  mechanized  infantry  tactics.  The  Red 
force  was  trained  to  use  accepted  Soviet  mechanized  infantry 
tactics. 

The  Blue  defensive  force  consisted  of  only  a  mechanized 
infantry  platoon  (i.e.,  the  Stingray  test  platoon).  The  Red 
offensive  force  consisted  of  a  mechanized  infantry  company  (using 
nine  BFVs  as  surrogates  for  the  Soviet  armored  infantry  vehicle, 
BMP)  supported  by  Soviet  tanks  (using  four  M-l  tanks  as 
surrogates  for  Soviet  main  battle  tanks) . 

The  Blue  offensive  force  consisted  of  three  M-l  tanks  and 
nine  BFVs.  Four  of  the  BFVs  were  those  assigned  to  the  Stingray 
test  platoon.  The  Red  defensive  force  consisted  of  a  mechanized 
infantry  platoon  (using  three  BFVs  as  BMP  surrogates)  supported 
by  a  Soviet  tank  (using  one  M-l  tank  as  a  surrogate  for  a  Soviet 
tank) . 

Each  BFV  assigned  to  the  Stingray  test  platoon  had  a  three- 
man  crew.  All  test  platoon  crew  members  were  qualified  as 
military  occupational  specialty  (MOS)  19D  cavalry  scouts.  The 
duty  positions  and  ranks  of  the  three  BFV  crew  members  were: 
Bradley  commander  (BC) ,  either  a  staff  sergeant  (E6)  or  a 
sergeant  (E5) ;  gunner,  a  specialist  or  corporal  (E4)  or  a  private 
first  class  (E3) ;  and  driver,  a  private  (E2) . 

Prior  to  the  start  of  the  test,  the  four  Stingray  test  crews 
received  classroom  and  field  training  on  Stingray  operations  and 
maintenance  from  the  system  contractor.  The  Stingray  platoon 
leader  and  platoon  sergeant  (who  were  not  assigned  to  one  of  the 
four  test  crews) ,  as  well  as  the  test  analysts  and  data 
collectors,  also  participated  in  Stingray  training. 


CEP  Test  Plan 

The  CEP  test  plan  called  for  a  series  of  free-play  offensive 
and  defensive  force-on-force  missions  conducted  under  both 
daylight  and  night  conditions.  During  each  mission,  the  Blue 
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force  was  either  in  a  baseline  condition  or  in  a  Stingray- 
augmented  condition.  During  Stingray  missions,  the  Stingray 
system  was  operated  in  one  of  its  three  different  modes  (i.e., 
automatic,  semiautomatic,  or  manual) .  Consequently,  the  test 
design  could  be  represented  as  a  2  X  2  X  4  factorial  experiment 
comprised  of:  (a)  day  or  night  operations;  (b)  offensive  or 
defensive  missions;  and  (c)  baseline.  Stingray-automatic, 
Stingray-semiautomatic  or  Stingray-manual  conditions. 

Two  BFVs  were  designated  as  test  vehicles  and  were  fitted 
with  special  instrumentation  (e.g.,  video  and  audio  recording 
devices) .  One  of  these  BFVs  was  equipped  with  the  Stingray 
system  to  be  used  in  Stingray  missions.  The  second  BFV  was  used 
in  the  baseline  missions  for  control  purposes.  Hence,  for  each 
mission,  three  of  the  test  crews  operated  their  usual  BFVs  and 
one  crew  operated  either  the  Stingray  BFV  or  the  designated 
baseline  BFV.  The  four  test  crews  rotated  through  the  Stingray 
and  baseline  BFVs.  Each  test  crew  was  to  participate  at  least 
once  in  each  of  the  16  test  conditions. 


Measures  of  System  and  Crew  Performance  and  Effectiveness 

Various  dependent  measures  were  to  be  used  to  determine  any 
difference  in  Blue  force  performance  and  effectiveness  between 
the  baseline  and  the  Stingray-augmented  mission  conditions. 

Since  Stingray  is  designed  to  protect  the  friendly  force  from 
fires  directed  by  enemy  electro-optical  sensing  and  aiming 
devices,  a  measure  of  Stingray's  effectiveness  is  the  force 
effectiveness  ratio  (FER) .  The  FER  is  the  proportion  of  enemy 
systems  "killed"  divided  by  the  proportion  of  friendly  systems 
killed  during  a  mission.  High  FER  values  indicate  that 
proportionally  fewer  friendly  systems  are  destroyed  than  enemy 
systems. 

Alternately,  FER  is  defined  as: 

LER 


IFR 


where, 


Loss  Exchange  Ratio  (LER) 


total  no.  of  enemy  forces  killed 

- - j  and 

total  no.  of  friendly  forces  killed 


initial  total  no.  of  enemy  systems 

Initial  Force  Ratio  (IFR)  =  . 

initial  total  no.  of  friendly  systems 


10 


Initial  numbers  of  systems  for  the  two  opposing  forces  were 
predetermined  for  each  test  mission.  The  numbers  of  systems 
killed  for  each  of  the  two  opposing  forces  were  determined  from 
data  obtained  from  Multiple  Integrated  Laser  Engagement  System 
(MILES)  instrumentation  on  each  vehicle. 

Crew  performance  measures  which  were  to  be  used  included 
time  and  error  measurements  for  target  engagement  procedures  that 
are  outlined  in  the  Bradley  Fighting  Vehicle  Crew  Drills 
(Training  Circular  7-8,  1985).  For  example,  the  time  from  target 
detection  to  engagement  can  be  obtained  from  target  handoff 
procedures  between  the  BC  and  gunner.  These  performance  data 
were  to  be  derived  from  video  and  audio  tapes  of  crew  actions 
during  Stingray  and  baseline  test  missions. 


Workload  Assessment  Instruments  and  Methods 


During  all  missions,  data  were  collected  to  determine  any 
impact  of  Stingray  on  the  capability  of  the  BFV  crew  to  perform 
their  required  tasks.  The  techniques  used  to  obtain  these  data 
included  operator  workload  ratings,  post-mission  debriefings, 
video  and  audio  recordings,  and  an  end-of-test  questionnaire. 

Task  Load  Index  (TLX)  technique.  The  TLX  technique  for 
assessing  workload  was  developed  by  the  NASA-Ames  Research  Center 
(1986) .  Ratings  of  workload  using  the  TLX  are  obtained  on  a 
scale  from  0  to  100  (low  to  high  workload)  for  each  of  the  six 
workload  rating  dimensions  which  comprise  the  TLX:  (a)  mental 
demand,  (b)  physical  demand,  (c)  temporal  demand,  (d)  perception 
of  performance,  (e)  effort,  and  (f)  frustration.  Information  on 
the  TLX  technique  is  presented  in  Appendix  A-l. 

The  TLX  was  used  in  this  study  to  obtain  absolute  judgments 
of  workload  from  the  BC  and  gunner  of  the  baseline  or  Stingray- 
augmented  BFV  crew.  The  procedure  for  obtaining  TLX  measures  is 
designed  to  account  for  differences  among  soldiers  in  their 
perception  of  workload  for  the  tasks  to  be  rated.  A  weighting 
procedure  requires  each  BC  and  gunner  to  designate  the  more 
relevant  dimension  of  workload  from  all  possible  pairs  of  the  six 
TLX  dimensions  (a  total  of  15  pairwise  comparisons) .  These 
paired  comparisons  were  obtained  for  target  acquisition  tasks  at 
the  conclusion  of  Stingray  training.  The  proportion  of  times 
each  workload  dimension  was  judged  to  be  more  relevant  than  the 
other  dimensions  was  used  to  weight  the  TLX  workload  ratings 
obtained  during  the  CEP  test.  A  unique  weighting  scale  was  thus 
developed  for  each  crew  member  and  used  in  the  analysis  of  the 
TLX  workload  data. 
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There  are  many  Bradley  crew  tasks  and  only  certain  tasks 
were  selected  for  workload  ratings.  With  assistance  from  subject 
matter  experts,  the  selection  of  tasks  was  based  on  their 
relevance  to  test  mission  scenarios  and  the  employment  of  the 
Stingray  system.  Six  tasks  in  the  target  acquisition  process 
described  in  the  Bradley  Fighting  Vehicle  Gunnery  Manual  (Field 
Manual  23-1,  1987)  were  selected  for  TLX  workload  ratings  by  the 
BC  and  gunner.  These  were: 

1.  Search  (look  for  targets), 

2.  Detection  (discovery  of  targets), 

3.  Location  (determine  target  position), 

4.  Identification  (recognize  target  type), 

5.  Classification  (prioritize  targets),  and 

6.  Target  handoff  (coordination  between  the  BC  and  gunner 

for  target  engagement  with  conventional  BFV  turret 
weapons) . 

Subjective  Workload  Dominance  (SWORD)  technique.  The  SWORD 
technique  for  assessing  workload  was  developed  by  Vidulich 
(1989) .  It  is  based  on  the  analytical  hierarchy  process  (AHP) 
developed  by  Saaty  (1980).  The  AHP  was  initially  applied  to  the 
assessment  of  workload  by  Lidderdale  (1987) .  SWORD  provides  a 
method  to  extract  expert  judgments  about  workload  for  tasks  in 
relation  to  other  tasks  or  to  evaluate  alternative  system 
designs.  Information  on  the  SWORD  technique  used  in  this  study 
is  presented  in  Appendix  A-2. 

The  SWORD  technique  was  used  in  this  study  to  elicit 
relative  judgments  about  workload  for  each  pairwise  combination 
of  the  baseline  and  three  Stingray  modes  of  operation.  The  six 
combinations  of  these  modes  of  operation  were  each  to  be  examined 
for  tasks  as  they  occurred  in  each  combination  of  mission  type 
(offensive  or  defensive)  and  time  of  day  (day  or  night) .  The  BCs 
and  gunners  were  to  indicate  whether  workload  for  a  particular 
task  and  mission  condition  was  equal  for  two  contrasting  modes  of 
operation  or  whether  it  was  higher  for  one  of  the  modes  of 
operation.  If  workload  was  judged  to  be  higher  for  one  mode  of 
operation,  the  rater  was  to  indicate  how  much  higher  on  an  8- 
point  scale.  For  example,  a  crew  member  could  indicate  that  the 
workload  associated  with  the  task  of  searching  for  targets  during 
daytime  offensive  missions  was  equal  for  the  Stingray- 
semiautomatic  and  baseline  modes  of  operation,  or  that  workload 
was  moderately  higher  (with  a  value  of,  say,  4)  for  the  Stingray- 
semiautomatic  mode  of  operation  chan  for  the  baseline  mode  of 
operation. 

The  SWORD  technique  was  used  only  to  obtain  prospective  and 
retrospective  ratings  of  workload.  In  both  cases,  the  same  six 
target  acquisition  tasks  that  were  selected  for  TLX  ratings  were 
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also  selected  for  SWORD  workload  ratings.  In  addition,  to  these 
six  target  acquisition  tasks  three  additional  tasks  were  selected 
for  retrospective  ratings  using  the  SWORD  technique.  The 
additional  three  tasks  were: 

1.  Preventive  maintenance,  checks,  and  services  (PMCS) , 

2.  Communications  (i.e.,  monitoring  and  inputting 

information  into  radio  nets) ,  and 

3.  Land  navigation. 

Post-mission  debriefs.  Post-mission  interviews  with  the  BC 
and  gunner  of  the  Stingray  or  baseline  BFV  were  conducted  by  the 
workload  analyst.  Comments  were  solicited  about  workload 
experienced  and  workload  strategies  employed,  task  performance 
and  task  performance  strategies,  system  and  operator  interface 
design,  and  issues  related  to  safety  and  health  hazards. 
Information  was  also  solicited  from  other  crews  to  elucidate  Blue 
and  Red  Force  tactics  and  performance. 

Video  and  audio  recordings.  Video  cameras  and  audio 
equipment  were  installed  in  the  Stingray  and  baseline  vehicle  to 
record  BC  and  gunner  actions  and  procedures  during  all  missions. 
The  recordings  were  reviewed  to  derive  time  and  error 
measurements  of  crew  tasks.  These  measures  of  crew  performance 
were  to  be  related  to  workload  ratings  and  system  performance 
measures.  The  recordings  were  also  used  to  gain  insights  about 
factors  affecting  crew  workload  and  performance. 

End-of-test  questionnaire .  A  questionnaire  was  developed 
based  on  the  Stingray  CEP  test  issues,  insights  acquired  during 
the  conduct  of  the  CEP,  and  various  other  concerns.  The  purpose 
of  the  questionnaire  was  to  elicit  crew  comments  based  on  their 
experience  with  Stingray  in  the  various  mission  conditions. 
Questionnaire  topics  included  task  performance  and  workload, 
safety,  training.  Blue  and  Red  force  performance  and  tactics,  and 
general  issues  about  Stingray.  A  copy  of  the  questionnaire  is 
presented  in  Appendix  A-3. 


Test  Procedure  for  Collecting  Workload  Assessment  Data 

The  Stingray  CEP  was  conducted  at  White  Sands  Missile  Range, 
New  Mexico,  from  January  8  through  March  12,  1990.  At  the 
beginning  of  the  evaluation  program,  the  four  BFV  crews  received 
four  days  of  classroom  training  and  two  days  of  practical 
exercises  on  Stingray  operations.  At  the  conclusion  of  this 
training,  the  concept  of  workload  and  workload  measurement 
techniques  were  discussed  with  the  test  platoon  leader,  platoon 
sergeant,  and  crews  (BCs,  gunners,  and  drivers) .  Prospective  TLX 
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and  SWORD  workload  ratings  of  the  designated  target  acquisition 
tasks  were  obtained  from  the  BCs  and  gunners.  These  ratings  were 
collected  to  evaluate  the  utility  of  these  instruments  for 
predicting  the  workload  that  the  BCs  and  gunners  would  experience 
in  test  missions. 

The  four  crews  then  participated  in  14  force-on-force  pilot 
study  missions.  Each  crew  operated  Stingray  in  at  least  two  of 
its  three  modes  during  either  a  defensive  or  offensive  mission. 
The  pilot  missions  were  conducted  primarily  to  permit  a  thorough 
evaluation  and  fine-tuning  of  the  methods  and  procedures  to  be 
used  by  test  players  and  field  test  administrators.  The  TLX 
workload  rating  scales  and  post-mission  debriefs  were 
administered  to  the  designated  BC  and  gunner  following  each  pilot 
mission.  This  was  done  to  refine  the  instruments  used  by  the 
analyst  and  to  familiarize  the  crews  with  the  data  collection 
procedures.  The  pilot  mission  workload  ratings  were  not  used  in 
the  analyses. 

Following  the  pilot  missions,  test  missions  "for  the  record" 
were  conducted.  In  rotation  throughout  the  test,  one  of  the  four 
trained  crews  was  designated  to  operate  the  baseline  or  Stingray 
vehicle.  Immediately  after  each  mission,  real-time  TLX  workload 
ratings  for  the  six  target  acquisition  tasks  were  obtained  from 
the  BC  and  gunner  of  the  designated  crew.  The  post-mission 
interviews  (debriefs)  with  these  crew  members  followed  completion 
of  the  workload  rating  scales.  Video  and  audio  tapes  of  the  BC 
and  gunner  from  the  designated  crew  were  also  recorded  during 
each  mission  to  obtain  operator  performance  data. 

On  the  day  following  the  last  test  mission,  retrospective 
TLX  and  SWORD  workload  instruments  were  administered  to  the  BCs 
and  gunners  of  the  four  BFV  crews.  The  six  target  acquisition 
tasks  were  retrospectively  rated  using  both  instruments;  three 
additional  tasks  were  retrospectively  rated  using  only  the  SWORD 
instrument.  The  four  crews  (including  drivers),  the  platoon 
sergeant,  and  platoon  leader  also  completed  the  end-of-test 
questionnaire.  In  addition,  the  troop  commander  completed  an 
abbreviated  version  of  the  questionnaire  primarily  covering 
tactics,  safety,  and  general  Stingray  issues.  Information 
obtained  from  the  debriefs,  tape  recordings,  and  questionnaires 
was  used  to  aid  in  interpreting  the  workload  ratings. 
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RESULTS 


Summary  of  Test  Conditions  and  Constraints 

Following  the  completion  of  the  Stingray  training  and  the  14 
pilot  test  missions,  50  "record"  test  missions  were  run.  Due  to 
requirements  imposed  on  the  conduct  of  the  test  and  the  realities 
of  field  testing,  not  all  missions  were  analyzable  and  a 
completely  counterbalanced  design  was  not  obtained.  After  the 
first  night  test  mission,  the  remaining  scheduled  night  missions 
were  canceled  because  of  safety  problems  associated  with  wearing 
laser  protective  lens  and  navigating  on  unfamiliar  terrain.  Two 
other  missions  were  aborted  for  various  reasons  before  they  could 
be  completed. 

As  a  result,  the  data  from  only  47  record  test  missions  were 
subjected  to  any  type  of  analysis.  Of  these,  real-time  TLX 
workload  ratings  were  obtained  and  analyzed  for  only  41  missions. 
All  crews  participated  at  least  once  in  every  test  condition, 
except  for  two  crews  who  did  not  operate  Stingray  in  the  manual 
mode  for  a  offensive  mission.  The  final  distribution  of  test 
conditions  over  the  four  test  crews  and  the  order  in  which  these 
conditions  were  administered  is  presented  in  a  matrix  in  Appendix 
Table  B-l .  There  is  no  evidence  to  suggest  any  systematic  bias 
occurred  while  rotating  the  four  test  crews  through  the  eight 
test  conditions. 


Real-Time  TLX  Workload  Ratings 

The  TLX  ratings  reflect  absolute  judgments  about  the 
magnitude  of  workload  on  a  numerical  scale  from  0  to  100.  Real¬ 
time  TLX  workload  ratings  (i.e.,  those  ratings  collected  at  the 
conclusion  of  each  record  mission)  were  subjected  to  a  mixed 
factorial  analysis  of  variance,  with  a  between-subjects  factor  of 
crew  position  (BC  or  gunner)  and  within-subjects  factors  of 
mission  type  (offensive  or  defensive) ,  mode  of  operation 
(baseline,  Stingray-automatic,  Stingray-semiautomatic,  or 
Stingray-manual) ,  and  task  (six  target  acquisition  tasks) .  Since 
there  were  unequal  numbers  of  observations  for  the  different 
combinations  of  test  conditions,  the  general  linear  model 
procedure  of  the  SAS  software  was  used  for  this  analysis  (see  SAS 
User's  Guide,  1985).  This  analysis  of  variance  show  that  the 
only  significant  source  of  variance  was  due  to  the  interaction  of 
mission  type  and  mode  of  operation,  F(3,17)  =  3.20,  p  =  0.05. 

(The  summary  of  this  analysis  of  variance  is  given  in  Appendix 
Table  B-2 ) . 
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Figure  1  shows  mean  real-time  TLX  ratings  for  mission  type 
and  mode  of  operation.  The  data  shown  in  the  figure  were 
obtained  by  averaging  across  crews,  crew  positions,  and  target 
acquisition  tasks.  As  shown  in  Figure  1,  crew  members  experience 
the  highest  level  of  workload  for  the  Stingray-manual  mode  of 
operation  during  offensive  missions.  They  report  the  lowest 
levels  of  workload  for  the  automatic  mode  of  Stingray  operations 
during  either  offensive  or  defensive  missions.  Ratings  of 
workload  for  the  remaining  five  test  conditions  were  at  about  the 
same  intermediate  level.  As  will  be  discussed  below,  this 
significant  interaction  effect  is  particularly  important  in  terms 
of  other  information  gained  from  the  crews  and  from  the  analysis 
of  force  effectiveness. 

While  no  significant  differences  were  found  for  target 
acquisition  tasks  or  for  crew  positions,  the  trends  demonstrated 
in  Figure  1  for  the  effect  of  mission  type  and  mode  of  operation 
are  evident  also  across  all  levels  of  both  of  these  two  factors 
(see  Appendix  Tables  B-3  and  B-4) .  In  general,  for  both  BCs  and 
gunners  and  for  all  six  target  acquisition  tasks,  workload 
ratings  were  lowest  for  the  automated  mode  of  operations  and 
highest  for  manual  operations  during  offensive  missions. 

Finally,  the  real-time  TLX  ratings  were  also  examined  for 
any  differential  contribution  of  the  six  TLX  subscales  to  the 
overall  workload  score.  No  significant  effects  for  the  subscale 
dimensions  were  found  for  any  of  the  test  conditions.  Mean 
subscale  ratings  are  presented  in  Appendix  Table  B-5. 


Workload  Strategies 

Information  required  to  identify  strategies  the  BFV  crew 
members  used  to  manage  workload  was  obtained  from  analyses  of  the 
data  derived  from  the  post-mission  debriefs,  video  and  audio 
recordings,  and  the  end-of-test  questionnaire.  Collectively, 
these  sources  of  information  show  clearly  that  the  crews  did 
develop  some  innovative  techniques  and  procedures  both  to 
compensate  for  the  problems  and  to  exploit  opportunities  created 
by  adding  Stingray  into  the  BFV  system. 

However,  the  very  fact  that  these  strategies  were  developed 
and  that  they  evolved  over  successive  missions  in  the  field  test 
caused  the  data  to  become  less  structured  and  less  quantifiable 
than  desired.  More  specifically,  the  strategies  developed  by  the 
BFV  crews  to  deal  with  Stingray-related  task  demands  also  caused 
the  performance  of  the  crew  members  to  progressively  deviate  from 
that  normally  dictated  by  standard  BFV  operating  procedures. 
Consequently,  the  crew  performance  data  that  were  derived  from 
the  video  and  audio  recordings  and  the  information  obtained  from 
crew  members  during  post-mission  debriefings  were  not  consistent 
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MEAN  TLX  REAL-TIME  WORKLOAD  RATINGS 


Figure  1 .  Mean  TLX  real-time  workload  ratings  by  mission  type  and 
mode  of  operation. 


over  successive  missions.  Because  of  the  dynamic  changes  in  crew 
techniques  and  procedures,  the  objectives  and  content  of  the  end- 
of-test  questionnaire  also  evolved  over  the  field  test  and  was 
finalized  only  at  the  end  of  the  test. 

Because  of  the  difficulty  encountered  in  summarizing  these 
data,  they  are  not  reported  in  this  section  of  the  report. 
Instead,  because  the  information  derived  from  these  data  depends 
on  interpretations  provided  by  the  authors,  this  information  is 
presented  in  the  discussion  section  which  follows.  It  aids  in 
clarifying  the  meaning  of  quantitative  workload  ratings  and 
performance  data. 


Workload  and  System  Performance 

The  relationship  between  the  workload  experienced  by  system 
operators  and  the  overall  performance  of  the  system  was  examined 
by  calculating  the  correlation  between  real-time  TLX  workload 
ratings  for  target  acquisition  tasks  and  the  force  effectiveness 
ratio  (FER) .  The  correlation  between  workload  ratings  and  system 
performance  is  significant  for  offensive  missions  (N  =  227,  r  =  - 
.35,  p  =  .0001);  the  correlation  was  nearly  significant  for 
defensive  missions  (N  =  256,  r  =  -  .12,  p  =  .0518).  In  general, 
higher  levels  of  BC  and  gunner  workload  for  target  acquisition 
tasks  are  associated  with  lower  levels  of  force  effectiveness. 

The  data  obtained  for  offensive  missions  best  illustrates 
the  nature  of  the  relationship  between  workload  and  system 
performance.  These  data  were  summarized  and  are  illustrated  in 
Figure  2.  During  offensive  missions  the  BCs  and  gunners 
experience  the  greatest  workload  and  FER  is  lowest  for  the  manual 
mode  of  Stingray  operations.  Least  workload  and  the  highest  FER 
are  recorded  for  the  automatic  mode  of  Stingray  operations. 
Intermediate  levels  of  workload  and  FER  are  recorded  for  the 
baseline  and  Stingray-semiautomatic  conditions.  During  defensive 
missions  the  relationship  between  workload  and  system  performance 
is  much  less  dramatic.  As  was  reported  previously  in  conjunction 
with  Figure  1,  there  was  little  variation  in  workload  ratings 
during  defensive  missions.  There  was  a  corresponding  absence  of 
variation  in  FER  for  defensive  missions.  (Note  that  FER  values 
are  classified  and  are  not  presented  in  this  report) . 


Prospective  and  Retrospective  Workload  Ratings 

Both  TLX  and  SWORD  workload  ratings  for  target  acquisition 
tasks  were  collected  prospectively  (at  the  conclusion  of 
training,  but  prior  to  pilot  missions)  and  retrospectively  (on 
the  day  following  the  last  record  mission) .  Mean  prospective, 
real-time,  and  retrospective  TLX  ratings  of  each  crew  member  for 
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FORCE  EFFECTIVENESS  RATIO  (FER) 


NOTE:  FER  values  are  classified. 


Figure  2.  Relationship  between  workload  and  force  effectiveness 
for  offensive  missions. 


each  combination  of  mission  type  and  Stingray  operations 
condition  are  presented  in  Appendix  Table  B-6.  Mean  prospective 
and  retrospective  SWORD  ratings  by  task  are  presented  in  Appendix 
Table  B-7 .  Summary  means  for  prospective  and  retrospective  TLX 
and  SWORD  ratings  are  presented  for  comparison  with  real-time  TLX 
ratings  in  Appendix  Tables  B-8  and  B-9,  respectively. 

Comparisons  between  workload  ratings  obtained  real-time  and 
those  obtained  prospectively  and  retrospectively.  The  validity 
of  prospective  and  retrospective  workload  assessment  techniques 
was  evaluated  by  correlating  the  ratings  each  generated  with  the 
workload  experienced  during  the  missions,  as  reflected  in  the 
real-time  TLX  ratings.  The  correlation  coefficients  were 
calculated  using  the  mean  ratings  over  crew  members  for  each  of 
the  eight  combinations  of  mission  type  (offensive  and  defensive) 
and  Stingray  operations  condition  (Baseline,  Stingray-automatic, 
Stingray-semiautomatic,  and  Stingray-manual)  and  hence  are  based 
on  a  sample  size  of  N  =  8. 

Prospective  and  retrospective  TLX  ratings  were  compared 
directly  to  the  real-time  TLX  ratings.  The  correlation  between 
prospective  and  real-time  TLX  rating-  was  not  significant 
(r  =  .18,  p  >  .69).  However,  there  is  a  significant  correlation 
between  retrospective  TLX  ratings  and  real-time  TLX  ratings 
(r  =  . 87 ,  p  <  . 01) . 

Since  the  SWORD  technigue  yields  relative  measures  of  the 
workload  experienced  for  each  major  test  condition  and  TLX  yields 
absolute  measures,  real-time  TLX  ratings  were  normalized  for 
comparison  to  the  SWORD  ratings.  The  correlation  between 
prospective  SWORD  workload  ratings  and  real-time  TLX  workload 
ratings  was  not  significant  (r  =  .61,  p  >  .11).  However, 
retrospective  SWORD  ratings  were  significantly  correlated  with 
real-time  TLX  ratings  (r  =  .83,  p  <  .01). 

Other  retrospective  workload  ratings  using  the  SWORD 
technique.  SWORD  ratings  were  also  obtained  retrospectively  for 
BFV  tasks  other  than  the  target  acquisition  tasks.  These  tasks 
were  preventive  maintenance,  checks  and  services  (PMCS) , 
communications,  and  land  navigation.  The  retrospective  SWORD 
workload  ratings  for  each  of  these  three  tasks  were  subjected  to 
a  separate  analysis  of  variance.  Factors  in  each  analysis 
included  crew  member  position  (BC  or  gunner)  and  mode  of 
operation  (Stingray-automatic,  Stingray-semiautomatic,  Stingray- 
manual,  or  Baseline). 

Significant  differences  among  the  modes  of  operation  were 
found  for  only  the  communications  and  land  navigation  tasks.  For 
the  communications  task.  Stingray-manual  mode  was  retrospectively 
rated  relatively  higher  in  workload  than  the  Stingray-automatic 
and  baseline  modes  of  operation  but  none  of  these  three 
conditions  had  a  relative  workload  rating  which  differed  from  the 
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Stingray-semiautomatic  mode  of  operation.  For  land  navigation, 
all  three  Stingray  modes  of  operation  were  rated  relatively 
higher  in  workload  than  the  baseline  condition  but  they  did  not 
differ  from  each  other.  A  summary  of  these  findings  is  given  in 
Appendix  Table  B-10. 


Evaluation  of  Workload  Assessment  Techniques 

The  techniques  employed  during  the  Stingray  CEP  included 
operator  workload  ratings,  post-mission  debriefs,  video  and  audio 
recordings,  and  an  end-of-test  questionnaire.  The  post-mission 
debriefs  proved  to  be  a  valuable  source  of  information  for 
understanding  the  impact  of  the  Stingray  system  from  the 
perspective  of  the  BFV  crew  members.  The  debriefings  conducted 
early  in  the  CEP,  to  include  those  conducted  following  each  pilot 
mission,  provided  valuable  information  about  some  misconceptions 
by  the  crews  about  Stingray  operations  and  how  the  system  was  to 
be  utilized.  Time  was  spent  with  the  crews  to  clarify  Stingray 
operations  and  the  type  of  information  that  was  being  sought  for 
the  workload  analysis.  The  qualitative  information  gained  from 
the  debriefs  was  used  in  the  workload  discussion  section  and  in 
the  training  section  of  the  Stingray  CEP  final  test  report 
(Williford  &  Wade,  1990) . 

However,  as  was  previously  reported  in  presenting  the 
results  for  workload  management  strategies,  the  data  derived  from 
the  video  and  audio  recordings  and  the  post-mission  debriefings 
were  not  sufficiently  consistent  over  successive  missions  to 
yield  reliable  results.  These  data  are  therefore  not  reported  in 
this  results  section.  Instead,  a  discussion  of  the  usability  and 
usefulness  of  these  various  techniques  for  assessing  and 
interpreting  workload  is  presented  in  the  next  section.  The 
verbatim  responses  of  all  test  participants  to  the  end-of-test 
questionnaire  are  presented  in  Appendix  C. 


DISCUSSION 

One  objective  of  this  research  was  to  provide  information  of 
immediate  practical  value  to  the  developers  and  proponents  of  the 
Stingray  system.  Proof  that  this  objective  was  met  is  attested 
to  by  the  fact  that  the  procedures,  findings,  and  recommendations 
generated  by  this  research  effort  became  integral  elements  of  the 
Stingray  CEP  plans,  execution,  and  test  report.1 


1  In  recognition  of  the  success  of  her  efforts  in  behalf  of  the 
Stingray  CEP,  the  senior  author  received  a  special  commendation 
and  reward  from  the  President  of  the  Air  Defense  Artillery  Board 
of  the  U.S.  Army  TEXCOM. 
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Meeting  this  first  objective  of  the  present  research  effort 
not  only  speaks  well  of  the  abilities  and  motivation  of  the 
research  team,  it  also  strongly  endorses  the  totality  of  the 
methods  employed,  from  the  beginning  of  the  test  planning  period, 
through  the  training  and  pilot  missions,  to  the  completion  of  the 
effort.  These  methods  ensured  that  the  workload  analyst  and  data 
collectors  became  fully  functioning  members  of  the  Stingray  CEP. 
Such  a  totality  of  involvement  with  the  larger  test  community 
must  be  considered  essential  whenever  there  is  a  desire  to  meet 
the  needs  of  the  "customer."  More  on  this  point  will  be 
discussed  later  in  this  section. 

The  second  major  objective  of  this  research  was  to  generally 
expand  and  enhance  the  Army's  capability  to  assess  and  evaluate 
the  impact  of  operator  workload.  The  achievement  of  this  second 
objective  was  guided  by  five  research  questions,  derived  from 
descriptions  of  an  equal  number  of  unresolved  workload  issues 
initially  given  in  the  introductory  section.  Succeeding  sub¬ 
sections  discuss  the  results  of  this  research  as  they  address 
each  of  the  five  research  questions. 


What  Is  the  Impact  of  Stingray  Operations  on  the  Workload  Experienced  bv  the  Bradley 
Commander  and  Crew? 

Operator  ratings  indicated  that  the  workload  experienced  by 
the  Bradley  commander  and  gunner  was  altered  by  the  addition  of 
the  Stingray  system  to  the  BFV.  A  more  thorough  understanding  of 
these  workload  ratings  was  obtained  by  integrating  them  with  the 
information  obtained  from  the  post-mission  debriefs,  video  and 
audio  recordings,  and  the  end-of-test  questionnaire.  These 
latter  sources  of  information  also  permitted  an  examination  of 
the  workload  associated  with  tasks  performed  by  the  driver  of  the 
Stingray-equipped  BFV. 


Bradley  commander  workload  for  target  acquisition  tasks 

The  results  show  that  the  workload  reported  by  the  BC  for 
target  acquisition  tasks  was  dependent  on  the  Stingray  mode  of 
operation  and  the  mission  type.  The  three  different  modes  of 
Stingray  operation  require  different  types  and  amounts  of  BC 
actions.  The  two  types  of  missions  reflect  the  effects  of 
operating  Stingray  when  the  host  BFV  was  on  the  move  (during 
offensive  missions)  or  stationary  (during  defensive  missions) .  A 
more  thorough  discussion  of  the  workload  impact  on  the  BC  of  each 
Stingray  mode  of  operation  during  performance  of  the  target 
acquisition  tasks  is  presented  in  succeeding  paragraphs. 


Stingray-manual  mode.  The  manual  mode  of  Stingray 


22 


operations  require  the  BC  to  perform  many  tasks  that  are  not 
normally  associated  with  commanding  a  BFV.  These  tasks  include 
controlling  the  search  pattern  of  the  STINGRAY  with  the  joystick, 
detecting  and  evaluating  targets  shown  in  the  Stingray  visual 
display,  and  countermeasuring  threats  with  a  trigger  pull.  As 
shown  in  Figure  1  and  Table  B-4,  the  manual  mode  of  Stingray 
operation  tended  to  be  associated  with  greater  workload  than  the 
other  modes  of  Stingray  operation  and  the  baseline  condition, 
particularly  while  the  host  vehicle  was  moving  during  offensive 
missions.  Even  while  seated,  if  the  BFV  was  moving  over  rough 
terrain,  it  was  difficult  for  the  BC  to  manually  search  the  VDU 
for  targets  or  to  use  the  joystick  to  place  the  cross  hairs  on  a 
target  for  countermeasure.  However,  the  BCs  were  rarely  seated 
and  tended  to  perform  most  of  their  assigned  tasks  while  standing 
up  in  the  turret,  with  much  of  the  upper  body  protruding  out  of 
the  hatch. 

This  upright  stance  was  assumed  by  the  BCs  to  improve  their 
capability  to  perform  command  and  control  (C2)  tasks  such  as 
those  associated  with  situational  assessment  and  land  navigation. 
The  BCs  reported  that  manually  operating  the  Stingray  system  is 
extremely  difficult  when  "out  of  the  hatch."  Some  BCs  tried  to 
concurrently  perform  Stingray  and  BFV  C2  tasks  by  riding  at 
"name-tag  height"  (i.e.,  halfway  out  of  the  hatch).  This  action 
did  not  solve  the  problem. 

The  out-of-the-hatch  operations  performed  by  the  BCs  were 
the  subject  of  much  discussion  and  controversy  during  the  CEP 
test.  Some  test  personnel  advocated  that  the  BC  operate  the  BFV 
and  the  Stingray  system  while  seated  with  the  hatch  securely 
closed  ("buttoned-up") ,  as  would  be  the  case  when  troops  are 
under  enemy  fire.  However,  the  Bradley  Fighting  Vehicle  Gunnery 
Manual  (FM  23-1,  1987)  states,  "During  buttoned-up  operations, 
the  Bradley  commander's  and  gunner's  ability  to  acquire  targets 
is  reduced  by  at  least  50  percent,  and  their  acquisition 
responsibilities  must  be  altered"  (p.  3-2) . 

While  the  Stingray  VDU  presents  the  BC  with  a  view  of  the 
battlefield  other  than  that  provided  by  the  integrated  sight  unit 
( ISU)  or  the  vision  blocks  installed  in  the  hatch,  it  still 
presents  the  BC  with  only  a  "slice  of  the  pie."  The  BCs 
reported  that  in  order  to  effectively  command  and  control,  they 
need  to  be  out  of  the  hatch  to  get  the  "big  picture"  of  the 
battlefield,  even  when  under  enemy  fire. 

Stingray-semiautomatic  mode.  The  results  show  that  the 
semiautomatic  mode  of  Stingray  operations  imposed  the  same  level 
of  workload  on  the  BCs  as  the  baseline  condition  (see  Table  B-4) . 
While  in  the  semiautomatic  mode,  Stingray  automatically  searches 
for,  detects,  and  locates  targets,  but  the  BC  still  must  manually 
perform  the  tasks  of  target  identification,  classification,  and 
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countermeasure.  Furthermore,  the  BC  must  respond  to  a  Stingray 
target  detection  signal  (via  an  auditory  alarm)  within  a  ten 
second  window.  Typically,  upon  receiving  the  detection  signal, 
the  BC  drops  down  into  his  seat  from  out  of  the  open  hatch.  If 
the  detected  target  displayed  on  the  VDU  is  judged  to  be  a 
threat,  the  BC  manipulates  the  joystick  to  place  the  cross  hairs 
on  the  target  and  depresses  the  countermeasure  trigger. 

Apparently,  the  reduction  in  workload  caused  by  automating 
the  search,  detect,  and  locate  tasks  is  sufficient  to  offset  any 
gain  in  workload  caused  by  the  requirement  to  identify,  classify, 
and  countermeasure  a  target  judged  to  be  a  threat.  The  BCs  did 
report  that  the  demands  of  the  semiautomated  Stingray  tasks  were 
made  more  difficult  during  offensive  missions  than  defensive 
missions  because  the  auditory  target  detection  alarm  was 
difficult  to  hear  over  the  noises  made  by  the  BFV  while  it  was  on 
the  move.  However,  the  BC  ratings  did  not  show  any  difference  in 
workload  due  to  mission  type  for  the  Stingray  semiautomatic  mode. 

Stingray-automatic  mode .  The  BCs  rated  the  workload 
associated  with  Stingray  in  the  automatic  mode  of  operation  as 
less  than  that  associated  with  the  other  two  Stingray  modes  and 
the  baseline  condition.  This  reduction  in  workload  rating  for 
the  automated  mode  of  operations  occurred  for  both  offensive  and 
defensive  missions.  The  reduced  workload  was  particularly 
pronounced  for  the  target  acquisition  tasks  of  searching, 
detecting,  and  locating  targets  (see  Table  B-3) . 

The  real-time  ratings  of  workload  experienced  with  the 
automatic  mode  of  Stingray  operations  were  higher  than  what  the 
BCs  predicted  using  the  prospective  rating  technique  (see  Table 
B-8 ) .  The  BCs  indicated  at  the  end  of  training  (but  prior  to  the 
pilot  missions)  that  they  thought  Stingray  would  automatically 
take  care  of  the  threat.  However,  on  the  end-of-test 
questionnaire,  the  BCs  commented  that  Stingray  is  to  be 
considered  "an  extra  pair  of  eyes,"  not  a  replacement  for  the 
target  acquisition  responsibilities  of  the  BC.  One  BC  commented, 
"Automatic  mode  is  not  any  more  independent  of  the  BC  than  cruise 
control  is  for  a  driver"  of  an  automobile. 


Bradley  commander  and  gunner  workload  for  target  acguisition 

tasks 


The  rating  data  showed  that  the  Bradley  gunner  and  BC 
experienced  similar  shifts  in  workload  over  combinations  of 
Stingray  modes  of  operation  and  mission  types.  In  fact,  the 
gunner  tended  to  experience  more  workload  than  the  BC  over  these 
test  conditions  (see  Table  B-4) .  Information  derived  from 
sources  such  as  the  end-of-test  questionnaire  suggest  that  the 
workload  experienced  by  both  the  BC  and  gunner  during  performance 
of  the  target  acquisition  tasks  are  due  to  changes  these  crew 
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members  made  in  operating  procedures.  Three  of  these  changes  in 
target  acquisition  procedures  are  discussed  in  succeeding 
paragraphs . 

1.  Since  the  Stingray  system  automatically  provides  the 
gunner  with  target  range  and  azimuth  information,  there  is  little 
or  no  need  for  the  BC  to  verbally  give  this  target  position 
information  to  the  gunner  as  prescribed  in  BFV  target  handoff 
procedures.  As  a  result  of  this  ••automation"  of  target  handoff 
information,  the  workload  of  both  the  sender  (BC)  and  receiver 
(gunner)  of  that  information  is  similarly  affected  for  all  modes 
of  operations. 

2 .  During  semiautomatic  and  manual  operation  of  the 
Stingray  system  the  BC  must  give  priority  to  the  performance  of  a 
number  of  Stingray-specific  actions.  Consequently,  a  number  of 
tasks  normally  performed  by  the  BC  were,  instead,  performed  by 
the  gunner.  One  critical  task  whose  performance  was  shown  to  be 
transferred  from  the  BC  to  the  gunner  is  that  of  traversing  the 
turret  to  orient  the  BFV  conventional  weapon  systems  toward  the 
location  of  a  target  to  be  engaged. 

3.  With  Stingray  added  to  the  BFV,  the  role  of  the  BC  and 
gunner  may  change  during  the  performance  of  tasks  associated  with 
searching  for  targets,  target  detection,  and  target  prioritiza¬ 
tion.  Changes  in  the  demands  these  tasks  placed  on  the  BC  and 
gunner  are  discussed  in  the  next  three  paragraphs. 

(a)  Gunners  reported  that  they  searched  for  targets 
outside  their  assigned  sector  in  order  to  acquire  information 
provided  by  Stingray  for  the  rest  of  the  platoon  and  Blue  force. 
Normally,  the  gunner  will  use  the  ISU  to  scan  only  his  assigned 
sector  and,  concurrently,  orient  the  BFV  weapon  systems  in  the 
general  direction  of  suspected  or  known  threats  for  targets. 

When  in  the  automatic  and  semiautomatic  mode,  Stingray  searches 
sectors  aligned  with  the  horizontal  line  of  sight  of  the  25-mm 
gun.  Therefore,  by  expanding  his  sector  of  search,  the  gunner 
also  increased  the  size  of  the  sector  searched  by  Stingray.  One 
gunner,  for  example,  reported  that  he  was  scanning  out  of  his 
assigned  sector  so  that  Stingray  "had  a  full  view". 

(b)  Gunners  assisted  the  BC  in  detecting  targets  for 
Stingray  countermeasure.  When  a  gunner  detected  targets  out  of 
the  Stingray's  field  of  regard,  he  would  relay  this  information 
to  the  BC.  Given  this  information,  the  BC  could  use  a  sector 
switch  to  change  the  Stingray  search  pattern.  With  increasing 
amounts  of  experience  with  the  Stingray  system,  the  BFV  crews 
discovered  that  the  quickest  way  to  bring  Stingray  onto  the 
target  detected  by  the  gunner  was  to  have  the  BC  depress  the 
Stingray  "stop/scan"  switch  which  immediately  cages  the  Stingray 
on  the  25-mm  gun  line  of  sight.  This  was  an  innovation  in 
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Stingray  employment  identified  and  implemented  by  the  crews  in 
the  test  platoon  —  not  a  procedure  included  in  Stingray 
training. 


(c)  With  Stingray,  the  gunner  (rather  than  the  BC)  may 
need  to  establish  target  priority,  particularly  in  a  multiple 
target  scenario.  The  Stingray  system  prioritizes  targets  on  the 
basis  of  only  their  range,  not  by  type  of  threat.  Normally,  the 
key  factor  in  determining  the  priority  for  engagement  is  the 
target's  eminent  threat  to  friendly  force  survival.  For  example, 
a  threat  tank,  with  its  greater  killing  capability,  has  a  higher 
engagement  priority  thar  a  threat  mechanized  infantry  vehicle, 
say,  a  BMP.  This  is  t*  ^e  even  if  the  BMP  is  at  a  shorter  range. 
Consequently,  the  gunner  may  not  rely  solely  on  the  order  of 
target  presentation  in  the  Stingray  range  and  azimuth  display. 
Stingray's  capabilities  also  may  alter  target  prioritization  in 
other  ways.  For  example,  in  one  mission  the  gunner  reported  that 
there  were  two  targets  in  view.  The  gun  of  one  target  was  facing 
towards  the  Stingray  BFV  and  that  of  the  other  target  was 
oriented  away  from  the  Stingray  vehicle.  The  gunner  opted  to 
engage  the  one  "not  looking,”  figuring  that  Stingray  would  take 
care  of  the  one  "looking." 


Bradley  commander  and  gunner  workload  for  other  tasks 

Stingray  also  affected  the  workload  ratings  of  the  BC  and 
gunner  for  BFV  tasks  other  than  those  associated  with  target 
acquisition.  These  workload  data  were  obtained  from  the 
retrospective  administration  of  the  SWORD  technique  (see  Table  B- 
10) .  The  workload  associated  with  the  tasks  of  preventive 
maintenance,  checks,  and  services  (PMCS) ,  communications,  and 
land  navigation  is  discussed  in  succeeding  paragraphs. 

PMCS.  The  addition  of  the  Stingray  system  to  the  BFV 
requires  some  increase  in  the  numbers  and  types  of  PMCS  tasks. 
However,  the  crews  never  had  to  perform  any  of  these  new  PMCS 
tasks  during  the  CEP.  All  PMCS  and  other  maintenance  and  repair 
of  the  Stingray  system  were  performed  by  contractor  personnel. 
Thus,  the  retrospective  workload  ratings  assigned  to  the  PMCS 
task  did  not  vary  as  a  function  of  mode  of  operation.  The 
slightly  higher  ratings  for  the  three  Stingray  modes  of  operation 
over  the  baseline  condition  were  due  to  the  crews'  understanding 
of  additional  system  requirements,  and  not  on  actual  experience. 

r.a mauni cations .  The  workload  associated  with  BFV 
communication  tasks  tend  to  be  rated  higher  when  the  baseline  BFV 
was  augmented  with  Stingray.  This  increase  in  workload  was 
statistically  significant  for  the  Stingray-manual  mode  of 
operation.  This  effect  may  be  attributed  partly  to  a  persistent 
problem  encountered  with  the  headset  mounted  in  the  Combat 
Vehicle  Crewmember  (CVC)  helmet.  Because  the  CVC  headset  did  not 
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operate  correctly,  the  BC  was  required  to  hand-hold  a  receiver 
between  the  inside  of  the  helmet  and  his  ear.  Consequently,  when 
Stingray  was  operated  in  the  manual  mode,  the  BC  had  to  use  one 
hand  to  hold  the  receiver  to  his  ear  and  the  other  to  operate  the 
Stingray  joystick.  These  requirements  left  no  hand  free  for  the 
BC  to  stabilize  himself  while  he  was  out  of  the  hatch  or  while 
the  BFV  was  moving  over  rough  terrain  during  offensive  missions. 

The  workload  associated  with  BFV  communication  tasks  was 
also  rated  higher  under  Stingray  conditions  because  the  BCs 
engaged  in  more  communications.  The  BCs  reported  that  they  felt 
an  increased  emphasis  to  provide  spot  reports  due  to  Stingray's 
capability  to  detect  targets  earlier  and  at  greater  range.  The 
accurate  target  range  information  provided  by  Stingray  allowed 
the  Blue  force  to  withhold  engagement  until  the  BFV  weapons  were 
within  their  respective  firing  ranges. 

Land  navigation.  Retrospective  workload  ratings  for  the 
land  navigation  tasks  performed  by  the  BC  increased  when  the 
Stingray  system  was  operated,  especially  if  it  were  operated  in 
the  semiautomatic  or  manual  mode.  Since  the  BC  is  required  to 
perform  Stingray-related  tasks  inside  the  BFV  turret,  there  was  a 
decrease  in  the  amount  of  time  the  BC  could  spend  out  of  the 
hatch,  assessing  the  terrain  and  giving  directional  assistance  to 
the  driver. 


Driver  w^r'  load 

The  workload  imposed  on  the  crew  of  a  Stingray-enhanced  BFV 
extends  to  the  BFV  driver.  An  experienced  driver  may  be  critical 
for  effective  operation  of  Stingray  by  the  BC.  The  additional 
workload  imposed  upon  the  driver  of  a  Stingray-enhanced  BFV  is 
discussed  in  the  following  four  paragraphs. 

Land  navigation.  The  Stingray  system  increased  the  demand 
on  the  driver  to  demonstrate  land  navigation  skills.  It  has  been 
established  that  the  BC  must  be  either  inside  or  looking  inside 
the  BFV  turret  to  properly  operate  Stingray  in  its  semi-automatic 
and  manual  modes  of  operation.  This  fact  prevents  the  BC  from 
being  "out  of  the  hatch"  to  acquire  detailed  land  navigation 
information  to  pass  on  to  the  driver.  An  experienced  driver, 
able  to  make  more  independent  land  navigational  decisions,  is 
therefore  required  to  reduce  the  amount  of  time  the  BC  needs  to 
spend  out  of  the  hatch  for  navigational  purposes. 

Smooth  ride.  The  Stingray  system  increased  the  demand  on 
the  driver  to  move  the  BFV  over  the  smoothest  possible  terrain. 

It  has  been  established  that  during  cross-country  movement,  the 
BC  is  physically  thrown  around  within  the  turret,  especially  when 
standing  up  out  of  the  hatch.  This  physical  jarring  and  jolting 
increased  when  the  BC  is  attempting  to  operate  the  Stingray 
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system.  For  example,  the  BC  must  dedicate  one  hand  to 
manipulating  the  joystick  in  the  semiautomatic  or  manual  modes  of 
Stingray  operation.  This  requirement  leaves  one  less  hand  free 
to  stabilize  the  body.  The  crews  also  reported  that  there  were 
more  Stingray  system  failures  when  the  BFV  bounced  hard  over  the 
rough  terrain,  such  as  over  sand  dunes.  These  examples  clearly 
demonstrate  a  need  for  the  driver  to  select  and  move  the  BFV  over 
the  smoothest  route  possible. 

Vehicle  positioning.  The  Stingray  system  increased  the 
demand  on  the  driver  to  select  and  assume  level  stationary  battle 
positions.  Proper  positioning  of  the  BFV  is  important  if  the 
Stingray  system  is  to  operate  effectively.  The  Stingray  system 
used  in  the  CEP  test  was  aligned  with  the  “water  line"  of  the 
test  BFV  (i.e.,  with  the  hull  of  vehicle).  If  the  Stingray  BFV 
was  positioned  "hull  up"  then  the  Stingray  search  beam  would  be 
pointed  upward  toward  the  sky.  To  make  matters  worse,  the 
Stingray's  vertical  orientation  is  not  readily  apparent  to  the  BC 
in  automatic  and  semiautomatic  modes.  No  picture  of  the  search 
pattern  is  presented  on  the  Stingray  VDU  in  the  automatic  mode 
and  a  target  detection  is  necessary  before  a  picture  is  presented 
in  the  semiautomatic  mode.  Consequently,  the  driver  must  be  able 
to  quickly  park  the  Stingray  BFV  in  a  level  position. 

Stingray  power  requirements.  The  Stingray  system  tested  in 
the  CEP  was  powered  from  the  BFV  engine.  The  drivers  explained 
that  it  was  necessary  to  "rev  up"  the  host  BFV  engine  in  order  to 
provide  enough  power  for  Stingray  system  operations.  This 
resulted  in  a  demand  on  the  driver  to  monitor  and  properly 
maintain  adequate  sources  of  BFV  electrical  power.  They  also 
noted  that  the  increased  noise  and  exhaust  fumes  caused  by 
revving  up  the  BFV  engine  could  allow  the  enemy  to  identify  the 
Stingray-equipped  BFV  and  its  position. 


Workload  of  the  BFV  crew 


The  crew  of  the  BFV  augmented  with  Stingray  may  be  required 
to  provide  more  "soft  and  hard  kills"  than  other  BFV  crews  in  the 
platoon.  Since  Stingray  can  be  used  to  delay  enemy  fire,  the  BC 
may  try  to  countermeasure  (soft  kill)  all  possible  threats  so  the 
platoon  can  continue  its  advance  during  offensive  operations. 
Since  Stingray  can  rapidly  detect  and  provide  range  and  azimuth 
information  for  potential  targets,  the  gunner  in  the  host  BFV  may 
have  the  earliest,  most  accurate  target  data  necessary  to  destroy 
(hard  kill)  threat  systems  using  conventional  weapons.  These 
advantages  of  employing  Stingray  and  the  corresponding  increase 
in  workload  is  more  pronounced  in  a  multiple  target  field. 

One  gunner  commented,  "The  range  information  helps  the 
gunner  to  engage  targets  with  better  first  round  hit 
probabilities."  Also,  the  Stingray  platoon  leader  said  he  felt 
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•’bolder"  when  on  the  attack  with  Stingray  in  his  platoon.  He 
advanced  his  platoon  closer  to  the  objective  knowing  that 
Stingray  could  delay  fire  from  the  threat  while  the  platoon  laid 
down  a  base  of  fire  in  the  overwatch  position.  Thus,  there  may 
be  more  reliance  on  the  Stingray  crew  to  scan  with  Stingray  out 
of  their  assigned  sector,  detect  a  greater  number  of  targets,  and 
engage  targets  sooner  than  other  BFV  crews  in  the  force. 


What  Workload  Management  Strategies  Are  Employed  bv  the  Crew  of  the  Stingrav- 
Augmented  BFV? 

In  general,  when  faced  with  high  task  demands,  operators  may 
adopt  any  number  of  strategies  in  an  attempt  to  achieve  and 
maintain  acceptable  levels  of  workload  and  performance.  For  crew 
operations,  as  is  the  case  with  the  BFV,  tasks  may  be  reallocated 
among  crew  members  in  an  effort  to  redistribute  the  workload.  As 
discussed  in  the  previous  section,  the  requirement  to  operate  the 
Stingray  system  was  accompanied  by  changes  in  a  number  of  BFV 
standard  operating  procedures. 

The  workload  strategies  employed  by  the  Stingray  crew  were 
identified  through  information  obtained  from  crew  debriefs,  the 
end-of-test  questionnaire,  and  the  audio  and  video  recordings. 

An  analysis  of  this  information  clearly  shows  that  the  closely 
interwoven  functions  of  the  BC,  gunner,  and  driver  of  the  test 
BFV  were  modified  with  the  addition  of  the  Stingray  system. 

It  was  found  that  the  BC  shed  tasks  to  the  gunner  or, 
alternatively,  the  gunner  assumes  the  responsibility  to  perform 
tasks  that  are  normally  assigned  to  the  BC.  For  example,  the  BC 
is  trained  to  follow  a  standard  target  handoff  procedure  in  which 
he  directs  the  gunner  to  a  target  by  verbal  command  (i.e.,  the  BC 
gives  range  information  and  identifies  target  type)  and  traverses 
the  turret  to  face  the  threat.  However,  with  Stingray,  the  BC 
countermeasured  targets  with  the  system  which,  in  turn,  relayed 
target  position  information  directly  into  the  gunner's  display. 
Therefore,  the  gunner  can  and  does  traverse  the  turret  and  engage 
targets  without  any  formal  handoff  procedures.  Details  of  these 
task  modifications  are  presented  in  a  previous  section  on  Bradley 
commander  and  gunner  workload. 

The  driver  of  the  Stingray  vehicle  also  assumed  a  portion  of 
the  functions  and  tasks  normally  assigned  to  the  BC.  For 
example,  it  was  difficult  for  the  BC  to  both  operate  Stingray  in 
the  manual  or  semiautomatic  modes  and  give  navigation  information 
to  the  driver.  For  missions  using  either  of  these  two  Stingray 
modes,  the  BCs  gave  the  driver  less  specific  directional 
information.  As  a  result,  the  driver  assumed  more  responsibility 
for  selecting  routes  and  stationary  positions  that  gave  Stingray 
the  best  possible  windows  of  opportunity. 
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What  Is  the  Relationship  Between  Workload  and  System  Performance? 


A  relationship  between  real-time  workload  ratings  and  system 
performance  was  found  —  higher  levels  of  crew  workload  were 
associated  with  lower  levels  of  force  effectiveness.  Crews 
experienced  the  greatest  workload  in  the  Stingray  manual  mode 
that,  in  turn,  was  associated  with  the  lowest  force 
effectiveness  ratio  (FER) .  Least  workload  and  the  highest  FER 
were  recorded  for  the  automatic  mode  of  Stingray.  High  FER 
values  indicate  that  proportionally  fewer  friendly  vehicles  were 
destroyed  than  enemy  vehicles. 

Clear  relationships  between  workload  and  performance  are  not 
always  found.  It  is  suggested  that  such  a  relationship  was 
demonstrated  in  this  test  for  several  reasons.  First,  the 
operator  actions  required  by  the  three  operating  modes  of 
Stingray  were  differentially  demanding  enough  to  be  reflected  in 
the  workload  ratings.  Second,  the  tasks  selected  for  workload 
ratings  were  relevant  to  the  objectives  and  mission  of  the 
Stingray  system.  Third,  the  measure  of  system  performance  used 
(i.e.,  the  FER)  was  appropriate  since  Stingray  is  designed  to 
protect  friendly  forces  from  fires. 

Therefore,  the  present  study  clearly  demonstrates  that  a 
relationship  between  workload  and  system  performance  is  most 
likely  to  be  found  for  those  cases  in  which  the  following 
conditions  are  true: 

1.  Workload  ratings  reflect  differential  system  demands, 

2.  Tasks  selected  for  workload  ratings  impact  system 

performance,  and 

3.  System  performance  measures  are  related  to  designated 

system  objectives. 


What  Is  the  Relationship  Between  Both  Prospective  and  Retrospective  Workload  Ratings 
and  Real-Time  Workload  Ratings? 

One  of  the  objectives  of  the  present  research  was  to 
evaluate  the  utility  of  using  workload  rating  scales  as 
prospective  (i.e.,  predictive)  and  retrospective  measures  of  the 
"real-time"  experience  of  workload.  Improved  analytical 
techniques  are  required  to  predict  workload  early  in  system 
development  where  the  greatest  design  flexibility  is  available 
with  the  least  impact  on  system  cost.  Retrospective  measures  of 
workload  are  important  in  situations  in  which  the  opportunity  to 
collect  "real-time"  workload  data  is  hindered,  such  as  in  an 
operationally  significant  field  environment.  The  Stingray  CEP 


30 


provided  a  unique  opportunity  to  gather  workload  ratings 
prospectively,  real-time,  and  retrospectively  from  one  set  of 
subjects. 

Analyses  revealed  no  significant  correlation  between 
prospective  ratings  using  either  the  TLX  or  SWORD  instruments  and 
real-time  ratings  using  TLX.  The  failure  of  the  BCs  and  gunners 
to  successfully  use  either  absolute  or  relative  measurement 
scales  to  predict  real-time  ratings  of  workload  may  have  been  due 
to  the  Stingray  training  they  received.  That  training  clearly 
was  not  sufficient  to  provide  crew  members  with  the  expert 
knowledge  necessary  to  correctly  predict  variations  in  the 
workload  they  would  experience  operating  a  BFV  whose  capability 
was  augmented  with  a  Stingray  system. 

Hence,  while  these  soldiers  might  well  be  considered  subject 
matter  experts  on  all  or  most  aspects  of  BFV  operations,  they 
were  not  experts  on  the  operational  characteristics  of  the 
Stingray  system  nor,  most  importantly,  on  the  interaction  of 
Stingray  and  BFV  operating  characteristics.  Indeed,  there  is 
some  evidence  that  the  Stingray-specific  and  Stingray-in-a-BFV 
knowledge  of  the  crews  was  still  increasing  at  the  conclusion  of 
the  CEP  field  test.  The  question  of  the  types  and  amounts  of 
knowledge  needed  to  qualify  someone  as  a  subject  matter  expert 
was  not  raised  as  a  research  question  for  this  study.  However, 
it  is  a  question  which  must  be  answered  if  we  are  to  ask  an 
individual  to  make  reasonable  predictions  of  the  workload  (or  any 
other  attribute)  associated  with  the  operation  of  a  system. 

Significant  positive  relationships  were  found  for  the 
retrospective  workload  ratings  using  either  the  TLX  or  SWORD 
techniques  and  the  real-time  ratings  using  TLX.  The  results 
clearly  show  that  either  of  these  two  rating  techniques  can  be 
used  by  BFV  crew  members  to  quantify  the  workload  they 
experienced  during  an  earlier  period  of  time.  There  is  no 
evidence  to  suggest  that  the  memory  of  those  earlier  experiences 
decayed  or  was  inhibited  over  the  six  weeks  of  testing. 


How  Useful  Are  the  Various  Techniques  Employed  Purine  the  Stingray  CEP  for 
Assessing  and  Interpreting  Workload  and  Performance? 

TLX.  There  is  no  evidence  that  BFV  crew  members  had  any 
difficulty  using  the  NASA-TLX  workload  measurement  technique  to 
report  their  real-time  perception  of  workload.  The  results 
obtained  using  the  TLX  scale  show  face  validity  in  that  the  test 
conditions  which  were  clearly  different  in  the  demands  they 
imposed  on  the  operators  were  associated  with  correspondingly 
different  TLX  workload  ratings.  The  validity  of  TLX  was  also 
demonstrated  by  the  relationship  between  the  TLX  ratings  and  a 
measure  of  system  performance,  the  force  effectiveness  ratio. 
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The  prospective  and  retrospective  uses  of  the  TLX  technique 
were  discussed  in  the  previous  section  of  this  report.  It  is 
sufficient  to  note  here  that  prospective  and  retrospective 
applications  of  TLX  have  been  previously  reported  (see  Hill, 
Zaklad,  Bittner,  Byers, &  Christ,  1988,  and  Hill,  Byers,  Zaklad,& 
Christ,  1988,  respectively).  However,  the  present  study  is  the 
first  to  assess  the  validity  of  TLX  as  a  retrospective  and 
prospective  technique  through  correlations  with  real-time 
operator  ratings. 

SWORD.  SWORD  is  a  recently  developed  instrument  and  has 
been  used  primarily  in  studies  with  Air  Force  pilots.  The  Army 
BFV  crews  had  some  difficulty  understanding  the  SWORD  workload 
instrument  as  it  was  initially  administered  in  this  study  (i.e., 
at  the  conclusion  of  Stingray  training) .  When  analysts 
questioned  some  crew  members  about  inconsistencies  in  their 
ratings,  it  was  discovered  that  some  had  marked  ratings  the 
opposite  of  what  they  had  intended.  After  making  some 
modifications  in  the  prospective  workload  instrument  (i.e., 
changing  scale  labels  from  those  used  previously,  see  Appendix  A- 
2) ,  the  crews  were  able  to  use  SWORD  retrospectively  at  the 
conclusion  of  the  test  with  less  apparent  difficulty. 

However,  even  after  modifying  the  SWORD  instrument,  it  is 
concluded  that  SWORD  was  more  difficult  to  administer  and  the 
workload  ratings  it  produced  were  more  difficult  to  score  and 
analyze  than  TLX.  Furthermore,  there  is  no  evidence  that  the 
relative  measures  of  workload  produced  by  SWORD  have  any 
advantage  to  or  are  quantitatively  different  from  normalized  TLX 
scores.  In  the  present  study,  the  prospective  and  retrospective 
use  of  SWORD  and  TLX  yielded  essentially  identical  results. 

Debriefs.  The  after-mission  interviews  or  debriefings  that 
were  conducted  with  the  BFV  crews  yielded  much  information  of 
general  value  to  this  effort.  For  example,  after  pilot  test 
missions,  the  crews  were  debriefed  and  were  administered  workload 
rating  scales  just  as  they  would  be  during  the  record  missions. 
This  provided  an  opportunity  for  the  crews  and  the  test  team  to 
clarify  any  misconceptions  that  existed  about  Stingray  operation, 
force  tactics,  test  conduct,  and  data  collection.  Many  valuable 
bits  of  information  were  gained  during  this  "test  drive"  of 
Stingray  which  led  to  initial  insights  about  workload  issues  and 
refinement  of  debriefing  questions  by  the  workload  analyst. 

Once  into  the  test,  the  post-mission  debriefs  with  the  crews 
provided  vital  information  used  to  clarify  the  workload  ratings 
and  to  identify  the  workload  strategies  employed  by  the  crew.  In 
addition,  important  information  was  obtained  regarding  other 
concerns  of  the  CEP,  such  as  those  related  to  training,  tactics, 
safety,  and  human  engineering  design. 
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Video  and  audio  recordings.  Measures  of  crew  performance 
were  to  be  obtained  from  the  video  and  audio  recordings. 

According  to  prescribed  BFV  procedures,  there  is  a  progressive 
interactive  verbal  exchange  of  information  between  the  BC  and 
gunner,  beginning  with  a  target  alerting  command  by  the  BC  and 
continuing  to  the  gunner's  announcement  of  weapon  fire,  "on  the 
way."  Based  on  these  written  procedures,  it  was  anticipated  that 
performance  time  and  error  measures  could  be  extracted  from  the 
video  and  audio  recordings  of  crew  actions  during  a  mission. 

These  measures  were  then  to  be  related  to  the  ratings  of  workload 
and  measures  of  system  performance. 

However,  with  Stingray,  the  BC  and  gunner  either  eliminated 
or  only  partially  followed  the  standard  BFV  operating  procedures. 
This  may  have  been  due  in  part  to  the  capability  of  Stingray  to 
detect  and  locate  targets.  It  may  also  be  due  to  insufficient 
crew  training  on  how  to  integrate  Stingray  operations  into 
routine  BFV  task  procedures.  As  a  consequence  of  the  crews' 
deviation  from  accepted  BFV  operating  procedures,  there  were  no 
consistent  indicators  on  the  recordings  from  which  to  derive  crew 
performance  measures.  However,  the  video  and  audio  tapes  did 
provide  an  objective  record  of  crew  actions  which  served  to 
clarify  the  self-report  of  actions  by  the  crews. 


End-of-test  questionnaire.  The  structured  end-of-test 
questionnaire  provided  information  which  was  quite  valuable  for 
interpreting  information  available  from  other  sources.  The 
objectives  and,  correspondingly,  the  contents  of  the  end-of-test 
questionnaire  evolved  over  the  course  of  the  CEP  and  were 
finalized  towards  the  end  of  the  test. 

It  was  of  particular  importance  that  the  end-of-test 
questionnaire  obtain  information  from  the  crews  about  the  effects 
of  Stingray  on  task  performance.  This  importance  was  underscored 
as  a  result  of  the  crew-initiated  departure  from  BFV  standard 
operating  procedures  and  the  absence  of  reliable  crew  performance 
data  in  the  video  and  audio  recordings.  The  questionnaire  also 
contained  items  that  elicited  much  useful  information  from  the 
crews  concerning  topics  such  as  future  training  needs  (e.g., 
safety,  tactics,  tips  on  Stingray  operation)  and  desired 
modifications  to  the  system  (e.g.,  hardware  and  software  fixes). 
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CONCLUSION 


The  findings  obtained  in  this  research  provided  information 
of  immediate  practical  value  to  the  developers  and  proponents  of 
the  Stingray  system.  The  workload  assessment  findings  are 
helpful  not  only  in  pinpointing  areas  of  potential  difficulty 
(e.g.,  high  workload  for  Stingray-manual  in  offensive  missions) 
but  also  areas  of  benefit  (e.g.,  reduced  workload  for  Stingray- 
automatic)  .  This  information  can  be  used  to  guide  many  "soldier- 
in-the-system"  decisions.  For  example,  early  in  system 
development  some  problems  may  be  alleviated  through  human 
engineering  design  (e.g.,  yoke  Stingray  components  to  the 
gunner's  station  so  the  gunner  can  operate  the  system  when  the  BC 
has  other  task  demands) .  Evidence  of  workload  management 
strategies,  such  as  the  reallocation  of  tasks  between  the  BC  and 
gunner,  may  be  used  to  guide  new  training  doctrine  for  the 
employment  of  Stingray.  Workload  findings  may  drive  changes  in 
tactics,  such  as  guidance  to  generally  employ  the  automatic  mode 
of  Stingray  operations,  especially  during  attack  missions,  and  to 
use  the  manual  mode  of  operations  only  for  situations  which 
present  a  clear  danger  of  fratricide. 

Soldier-oriented  issues,  such  as  operator  workload,  do  not 
always  receive  the  attention  they  should.  However,  they  are  more 
likely  to  "speak"  to  the  decision  makers  when  they  are  tied  in 
with  system  performance.  In  the  Stingray  CEP,  it  was  clearly 
demonstrated  that  the  types  and  amounts  of  operator  actions 
required  by  each  of  the  operating  modes  of  Stingray  are 
associated  with  different  levels  of  workload  as  well  as  changes 
in  force  effectiveness.  It  is  important  to  stress  that  workload 
problems  noted  in  field  testing,  where  there  are  less  serious 
consequences  of  failure,  may  be  exacerbated  in  battlefield 
operations.  During  the  stress  of  battle,  the  same  workload 
problems  may  significantly  reduce  the  likelihood  of  a  successful 
mission.  It  is  these  types  of  findings  which  demand  attention 
from  the  decision  makers. 

This  study  also  provides  findings  of  immediate  practical 
value  to  those  charged  with  the  responsibility  for  assessing 
workload  and  interpreting  its  impact  on  soldier  and  system 
performance.  From  a  methodological  perspective,  this  study 
demonstrated  two  critical  and  interrelated  preconditions  for 
successful  field  studies  of  workload. 

First,  it  was  shown  that  a  thorough  workload  assessment 
relies  on  early  preparation  and  total  participation.  The 
workload  analyst  must  become  "system  smart"  and  develop  close 
working  relationships  with  all  the  other  test  personnel  and 
players.  The  workload  analyst  must  also  begin  gathering  workload 
information  as  soon  as  possible  in  order  to  gain  initial  insights 
and  refine  data  collection  techniques.  Since  a  field  test 


undergoes  many  changes  throughout  its  planning  and  execution 
phases,  the  analyst  must  be  flexible  in  how  to  best  obtain  the 
data  relevant  to  the  research  objectives.  For  example,  if  crew 
performance  (i.e.,  task  performance  time  and  error)  were  the  only 
workload  measure  planned  in  the  present  study,  the  assessment  of 
workload  in  this  study  would  have  been  severely  compromised.  It 
was  not  apparent  until  several  test  missions  had  occurred  that 
these  data  would  be  unobtainable. 

Second,  since  workload  is  a  function  of  many  variables,  it 
is  necessary  that  multiple,  redundant  techniques  and  sources  of 
information  be  employed  to  assess  workload.  This  is  particularly 
important  in  a  field  test  situation  which  is  characterized  by 
many  uncontrolled  variables  and  uncontrollable  conditions.  In 
this  type  of  environment,  the  total  workload  picture  can  only  be 
gained  through  the  integration  of  a  "patchwork"  of  bits  of 
information,  such  as  were  utilized  in  this  study  (i.e.,  operator 
workload  ratings,  post-mission  debriefs,  video  and  audio 
recordings,  and  a  post-test  questionnaire) . 

Another  conclusion  from  this  research  supports  the  position 
that  a  well  developed  workload  assessment  program  will  permit  an 
examination  of  the  workload  imposed  upon  a  crew  or  team  of 
individuals  while  they  jointly  operate  a  system.  The  capability 
to  analyze  crew  workload  permits  an  examination  of  the  effects  of 
crew  member  coordination  and  shared  task  performance  on  the 
management  of  workload  and  the  optimization  of  system 
performance. 

Finally,  this  research  supports  the  position  that  operators 
can  use  rating  techniques  retrospectively  to  report  the  workload 
they  experienced  during  an  earlier  period  of  time.  This 
conclusion  is  important  since  it  is  often  not  possible  to  obtain 
workload  ratings  during  or  immediately  following  a  period  of 
performance.  The  research  was  not  able  to  demonstrate  that 
operators  can  use  rating  techniques  prospectively  to  predict  the 
workload  they  would  experience  in  some  future  system  or  future 
operating  environment.  This  latter  result  raises  the  issue  of 
the  types  and  amounts  of  knowledge  and  experience  necessary  to 
qualify  someone  as  a  subject  matter  expert. 
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NASA  Task  Load  Index  (TLX) 


TLX  is  a  workload  assessment  technique  which  elicits 
absolute  judgments  of  workload  on  a  scale  from  0  to  100  (low  to 
high  workload).  It  is  comprised  of  six  workload  dimensions?  1) 
mental  demand,  2)  physical  demand,  3)  temporal  demand,  4) 
performance,  5)  effort,  and  6)  frustration. 

TLX  accounts  for  differences  among  individuals  in  their 
perception  of  workload  for  specific  tasks.  The  individual 
weighting  procedure  requires  the  rater  to  designate  the  more 
relevant  dimension  of  workload  from  all  possible  pairs  of  the  six 
dimensions  (a  total  of  15  pairwise  comparisons) .  In  the  present 
study,  crew  members  were  presented  with  15  separate  slips  of 
paper.  Each  slip  of  paper  listed  two  of  the  dimensions  (e.g., 
temporal  demand  and  mental  demand) .  The  crew  members  circled 
which  of  the  two  dimensions  contributed  more  to  the  experience  of 
workload  when  performing  target  acquisition  tasks.  The 
proportion  of  times  each  workload  dimension  was  judged  to  be  more 
relevant  than  the  other  dimensions  was  used  to  weight  the 
ratings.  A  unique  weighting  scale  was  thus  developed  for  each 
crew  member. 

Six  tasks  in  the  target  acquisition  process  were  rated  using 
the  TLX  scale.  These  tasks  were;  1)  search  for  targets,  2) 
detection  (discovery  of  targets) ,  3)  location  (determine  target 
position) ,  4)  identification  (recognize  target  type) ,  5) 
classification  (prioritize  targets) ,  and  6)  target  handoff 
(coordination  between  the  BC  and  gunner  for  target  engagement) . 
Each  of  these  tasks  was  rated  for  workload  demand  for  each 
combination  of  mission  conditions  (offensive  or  defensive)  and 
mode  of  operation  (baseline  or  one  of  the  three  modes  of  Stingray 
operation) . 

When  used  prospectively,  half  of  the  crewmembers  rated 
offensive  and  half  rated  defensive  missions.  This  was  done  to 
reduce  the  number  of  ratings  elicited,  since  prospective  ratings 
included  daytime  and  night  missions.  Since  night  missions  were 
cancelled  during  the  test,  retrospective  ratings  were  obtained 
from  all  the  crews  for  both  offensive  and  defensive  missions. 

The  prospective  and  retrospective  TLX  ratings  for  the 
target  acquisition  tasks  were  examined  for  possible  correlation 
with  the  real-time  TLX  ratings.  The  real-time  TLX  ratings  were 
analyzed  to  determine  any  mission  condition  effects  on  workload 
and  examined  for  any  differential  contribution  of  the  six 
subscale  dimensions  (e.g.,  effort,  frustration)  to  the  experience 
of  workload. 

For  illustration,  one  of  the  pages  from  the  TLX  real-time 
workload  rating  forms  follows. 
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Task  or  Mission  Segment:  mwinc*not  (recognize  target  type) 

Please  rate  the  task  or  mission  segment  by  putting  mark  on  each  of  the  six 
scales  at  the  point  which  matches  your  experience. 

Mental  . 

Demand 

0  100 
(HOW  MENTALLY  OEMANDlNG  WAS  THE  TASK?) 

Physical 
Demand 

u  100 

(HOW  PHYSIC  ALLY  OEMANDlNG  WAS  THE  TASK?) 

Temporal 
Demand 

o  100 

(HOW  HURRIED  OR  RUSHED  WAS  THE  PACE  OF  THE  TASK?) 

Performance 

r«rf«a  Fjilut* 

(HOW  SUCCESSFUL  WERE  YOU  IN  ACCOMPLISHING  THE  TASK?) 


o  100 

(HOW  INSECURE.  DISCOURAGED,  IRRITATED.  ANO  ANNOYED  WERE  YOU7) 
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Subjective  Workload  Dominance  (SWORD) 


SWORD  is  a  relative  workload  assessment  technique.  SWORD 
provides  a  method  to  extract  expert  judgments  about  workload  for 
tasks  in  relation  to  other  tasks  or  for  alternative  system 
designs.  All  possible  pairs  of  items  (tasks,  mission  conditions, 
etc.)  to  be  compared  are  presented  to  the  rater.  If  one  of  the 
pair  is  judged  to  have  higher  workload,  then  the  rater  judges  how 
much  higher  on  a  scale  from  1-8. 

The  comparisons  are  used  to  fill  a  judgment  matrix  in  which 
each  row  represents  one  item's  workload  dominance  relative  to  all 
of  the  other  items.  The  next  step  is  to  estimate  the  true  ratio 
scale  underlying  the  rater's  judgments.  Saaty  (1980)  proposed  a 
normalized  eigenvector  scale,  however  others  have  suggested  a 
geometric  means  approach  (Vidulich,  1989;  Williams  &  Crawford, 
1980*)  .  The  geometric  means  approach  may  be  more  compatible  with 
existing  statistical  techniques,  easier  to  understand  and  to 
calculate.  This  approach  was  used  in  the  present  study.  The 
data  can  then  be  subjected  to  statistical  analyses. 

After  observing  some  difficulty  by  the  crews  in  understanding 
the  scale  when  used  prospectively  (i.e.,  at  the  conclusion  of 
Stingray  training) ,  the  SWORD  instrument  was  modified  for  the 
retrospective  administration.  Verbal  anchors  for  the  scale  were 
changed  from  "absolute,  very  strong,  strong,  weak,  and  equal"  to 
the  more  appropriate  labels  of  "extremely  high  relative  workload, 
very  much  higher,  moderately  higher,  slightly  higher,  and  equal 
workload" . 

Crew  members  assessed  the  direction  and  magnitude  of  the 
difference  in  workload  for  each  task  under  all  possible  pairs  of 
mission  conditions.  The  mission  conditions  consisted  of  the 
three  modes  of  Stingray  operation  and  the  baseline  condition  for 
offensive  and  defensive  missions.  Tasks  included  the  six  target 
acquisition  tasks  (search,  detect,  locate,  identify,  classify, 
and  target  handoff)  and  other  BFV  tasks  of  PMCS  (preventive 
maintenance,  checks  and  services) ,  communications,  and  land 
navigation. 

The  prospective  and  retrospective  SWORD  ratings  for  the 
target  acquisition  tasks  were  examined  for  possible  correlation 
with  the  real-time  TLX  ratings.  The  retrospective  SWORD  ratings 
for  the  other  BFV  tasks  were  analyzed  to  determine  any  mission 
condition  effects  on  workload. 

The  example  sheet  and  one  of  the  pages  from  the  SWORD 
retrospective  rating  packet  is  included  for  illustration. 

Williams,  C.,  &  Crawford,  G.  (1980,  May).  Analysis  of  subjective  judgment  matrices 
(Technical  Report  R-2572-AF).  Santa  Monica,  CA:  Rand. 
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END-OF-TEST  QUESTIONNAIRE 


STINGRAY  QUESTIONNAIRE  NAME 


I.  TASK  PERFORMANCE  and  WORKLOAD 

1.  In  what  ways  has  STINGRAY  changed  how  you  accomplish  your 
tasks?  (e.g.,  have  you  dropped  any  less  important  tasks  in 
order  to  accomplish  STINGRAY  tasks,  have  other  crew  members 
picked  up  some  of  the  tasks  you  normally  do) 


2 .  Do  you  think  the  principal  operation  of  STINGRAY  should  be  in 
the  BC's  station  (as  it  was  tested)  or  should  it  be  moved  to 
another  crew  member's  (e.g.,  gunner)?  Why  or  why  not? 


3.  How  often  did  STINGRAY  pick  up  targets  that  you  had  not 
already  detected  by  other  means  (e.g.,  ISU,  reports  from 
others  in  Force) ? 


a )  Auto  mode 
never  ! 


very 

often 


b)  Semi-auto  mode 
never  ! 


very 

often 


c)  Manual  mode 
never  I 


very 

often 


4.  How  useful  was  the  range/azimuth  information  displayed  by 
STINGRAY  to  you  in  engaging  targets  with  BFV  conventional 
weapons? 

no  difference  very 

from  baseline  |  _  j  _  ]  _ _  j  _  [  _  ]  _  ]  useful 

missions 


5.  What  percentage  of  the  time  is  the  BC  commanding  out  of  the 
hatch? 

a)  on  the  attack  _ %  b)  on  the  defense  _ % 
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6.  What  percentage  of  the  time  are  you  looking  through  BFV 
optical  sights? 

a)  on  the  attack  _ %  b)  on  the  defense  _ % 

7.  Do  you  think  STINGRAY  affected  your  performance  during  the 
missions? 

Give  a  number  on  a  scale  of  -10  to  +  10,  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
+10  =  greatly  improved  my  performance 

a)  Overall  performance  of  my  duty  position 

ATTACK  DEFEND 

___  STINGRAY-AUTO  _  STINGRAY -AUTO 

_  STINGRAY-SEMI  _  STINGRAY-SEMI 

_  STINGRAY-MANUAL  STINGRAY-MANUAL 

COMMENTS : 

b)  Search  for  targets 

ATTACK  DEFEND 

_  STINGRAY-AUTO  _  STINGRAY -AUTO 

_  STINGRAY-SEMI  _  STINGRAY-SEMI 

_  STINGRAY -MANUAL  _  STINGRAY -MANUAL 

COMMENTS : 


c)  Detection  (discovery  of  targets) 
ATTACK 

_  STINGRAY-AUTO 

_  STINGRAY-SEMI 

_  STINGRAY-MANUAL 

COMMENTS : 


DEFEND 

STINGRAY-AUTO 
_  STINGRAY-SEMI 

STINGRAY-MANUAL 
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d)  Location  (determine  target  position) 

ATTACK  DEFEND 

_  STINGRAY-AUTO  _  STINGRAY -AUTO 

_  STINGRAY-SEMI  _  STINGRAY-SEMI 

_  STINGRAY-MANUAL  STINGRAY-MANUAL 

COMMENTS : 


e) 

Classification  (prioritize 

ATTACK 

targets) 

DEFEND 

STINGRAY-AUTO 

STINGRAY-AUTO 

STINGRAY-SEMI 

STINGRAY-SEMI 

STINGRAY -MANUAL 

STINGRAY-MANUAL 

COMMENTS : 

f) 

Identification  (recognize 

ATTACK 

target  type) 

DEFEND 

STINGRAY-AUTO 

STINGRAY-AUTO 

STINGRAY-SEMI 

STINGRAY-SEMI 

ST INGRAY -MANUAL 

STINGRAY -MANUAL 

COMMENTS : 

g) 

Target  handoff 

ATTACK 

DEFEND 

STINGRAY-AUTO 

STINGRAY-AUTO 

STINGRAY-SEMI 

STINGRAY-SEMI 

STINGRAY-MANUAL 

STINGRAY-MANUAL 

COMMENTS : 


A-ll 


h)  Employ  BFV  weapon  systems 


ATTACK 

_  STINGRAY-AUTO 

_  STINGRAY-SEMI 

_  STINGRAY-MANUAL 

COMMENTS : 

i )  Commo 

ATTACK 

_  STINGRAY-AUTO 

_  STINGRAY-SEMI 

_  STINGRAY-MANUAL 

COMMENTS : 

j )  Land  navigation 
ATTACK 

_  STINGRAY-AUTO 

_  STINGRAY-SEMI 

_  STINGRAY-MANUAL 

COMMENTS : 

h)  Command  and  control 
ATTACK 

_  STINGRAY-AUTO 

_  STINGRAY-SEMI 

STINGRAY-MANUAL 


DEFEND 

_  ST INGRAY -AUTO 
_  STINGRAY-SEMI 

STINGRAY -MANUAL 


DEFEND 

_  STINGRAY-AUTO 
_  STINGRAY-SEMI 

STINGRAY -MANUAL 


DEFEND 

_  STINGRAY-AUTO 
_  STINGRAY-SEMI 

STINGRAY -MANUAL 


DEFEND 

STINGRAY -AUTO 
_  STINGRAY-SEMI 

STINGRAY-MANUAL 


COMMENTS i 


II. 


SAFETY 


1.  Does  STINGRAY  pose  any  particular  health  or  safety  hazards  to 
user  personnel? 

2.  How  concerned  were  you  about  your  own  health  and  safety 
whenever  you  were  in  close  proximity  to  the  STINGRAY  vehicle? 

a)  during  the  conduct  of  this  test 


not 

concerned  very 

at  all  J  _  |  _  |  _  '  _  |  _  |  _  j  concerned 


b)  during  an  actual  battle  with  STINGRAY  at  its  full 
capabilities 

not 

concerned  very 

at  all  J  _  |  _  |  _  J  _  [  _  [  _  ]  concerned 


3.  Do  you  feel  the  safety  procedures  utilized  during  this  test 
need  any  improvement?  If  so,  then  please  explain. 


III.  TRAINING 


1.  How  would  you  rate  the  training  you  received  on  STINGRAY 
operations? 

Poor  ]  _  J  _  j  _  j  _  ]  _  ]  _  J  Excellent 

2.  How  would  you  rate  the  STINGRAY  operator's  manual? 

Poor  ]  _  !  _  j  _  J  _  j  _  |  _  |  Excellent 

3.  How  useful  was  the  condensed  version  of  STINGRAY  operator's 
manual? 

no  help  very 

at  all  ]  _  J  _  j  _  J  _  [  _  j  _  J  useful 
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4.  Do  you  have  any  comments  or  suggestions  about  how  to  improve 
the  STINGRAY  training  for  future  crews? 

5.  If  you  were  to  pass  along  any  tips  about  STINGRAY  operations 
and  functions,  what  would  they  be? 


6.  Do  you  feel  that  personnel  need  any  additional/specific 

skills  or  aptititudes  to  effectively  operate  STINGRAY  (other 
than  what  is  currently  required  for  your  MOS)? 


IV.  PERFORMANCE 


1.  In  general,  the  survivability  of  the  Blue  Force  was  due  to 


BLUE  BLUE 

ATTACK  DEFEND 


% 

% 

% 

% 

% 


%  presence/absence  of  STINGRAY 
%  tactics  employed 

%  personnel 

%  instrumentation  (e.g.,  MILES) 

%  other  _ 


NOTE:  these  should  add  up  to  100% 


2.  How  would  you  tactically  deploy  STINGRAY  on  the  defense  and 
attack? 

Please  include  your  recommendations  for  where  to  position  it 
in  the  formation,  how  many  STINGRAYS  should  a  platoon  or  a 
company  have,  what  mode  (Auto,  Semi,  Manual)  of  operation (s) 
would  be  best,  would  it  work  to  have  STINGRAY  in  the  Platoon 
Leader's  or  Sergeant's  or  even  Company  Commander's  vehicle? 

a)  on  the  defense: 

b)  on  the  attack: 

c)  any  other  comments  about  tactics: 

3.  How  useful  was  the  target  position  information  relayed  by  the 
STINGRAY  crew  to  the  rest  of  the  Blue  Force? 

no  difference  very 

from  baseline  ]  _  ]  _  j  _  j  _  |  _  |  _  J  useful 

missions 
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4.  Do  you  think  STINGRAY  accomplished  its  goal  of  delaying  the 
Red  Force's  fire  so  that  the  Blue  Force  could  engage  the 
enemy  with  BFV  conventional  weapons?  Why  or  why  not? 

5.  What  tactics  do  you  think  the  Red  Force  used  in  response  to 
STINGRAY?  (in  Red  Defend  and  Red  Attack  missions) 


6.  Do  you  think  STINGRAY  would  be  effective  on  any  other  types 
of  combat  vehicles  (e.g.,  tanks,  wheeled  vehicles,  air 
defense  systems) ? 


7.  Are  there  any  effects  of  the  environment  on  the  effective  use 
of  STINGRAY?  (e.g.,  weather,  terrain,  day/night  operations) 


8.  What  do  you  like  best  and  least  about  the  STINGRAY  system? 


9 .  What  changes  would  you  recommend  for  future  development? 


10.  How  effective  do  you  think  STINGRAY  would  be  in  an  actual 
battle? 

(with  its  full  capabilities  and  intended  functions) 


11.  Do  you  think  the  Army  should  buy  the  STINGRAY  system? 


ANY  OTHER  GENERAL  COMMENTS  OR  SUGGESTIONS: 
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APPENDIX  B 


DETAILED  RESULTS 
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B-l .  Test  mission  matrix. 

B-2.  ANOVA  table  for  TLX  real-time  workload  data. 

B-3 .  Mean  real-time  TLX  workload  ratings  for  target  acquisition 
tasks. 

B-4 .  Mean  real-time  TLX  workload  ratings  by  crew  position. 

B-5 .  Mean  real-time  TLX  workload  ratings  for  subscale 
dimensions. 

B-6.  Prospective,  real-time,  and  retrospective  TLX  workload 
ratings  for  each  BC  and  gunner. 

B— 7 .  Mean  prospective  and  retrospective  SWORD  workload  ratings 
by  task. 

B— 8 .  Mean  prospective,  real-time,  and  retrospective  TLX 

workload  ratings  by  mission  type  and  mode  of  operation. 

B-9 .  Mean  prospective  and  retrospective  SWORD  ratings  and  real¬ 
time  TLX  workload  ratings  by  mission  type  and  mode  of 
operation. 

B-10.  Retrospective  SWORD  workload  ratings  and  analyses  for 
PMCS,  communications,  and  land  navigation  tasks. 
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Table  B-l.  Test  mission  matrix. 


STINGRAY 

MODE  OF  OPERATION 

MISSION 

CREW 

TYPE 

BASELINE 

Auto 

Seal 

Manual 

Attack 

28 ,  64 

50 

49,  63 

1 

Defend 

17 

33,  41,  48 

32,  40 

21 

Attack 

29,  57 

30,  61 

60 

52 

2 

Defend 

42 

22,  36 

23,  37 

35 

Attack 

62 

56 

31,  55 

_ _ 

3 

Defend 

20 

15 

24,  45 

34,  38,  46 

Attack 

54 

51 

58 

59 

4 

Defend 

18,  47 

25,  19 

26,  44 

16,  43 

KEY:  Each  mission  in  the  test  was  numbered  in  consecutive  order.  Each  number  presented  in  this  table 
represents  a  single  record  mission  completed  by  the  designated  crew.  For  example.  Crew  1  operated 
Stingray  in  the  automatic  mode  for  three  defend  missions  (Nos.  33,  41,  48). 

NOTE:  A  total  of  64  missions  were  completed,  with  missions  1-14  representing  pilot  missions  (not  reported 
here).  Missions  19  and  53  were  aborted  and  Mission  27  was  conducted  at  night.  No  data  from  these  three 
missions  were  used  in  the  analyses.  Workload  ratings  were  not  obtained  for  6  missions  (these  underlined  in 
the  table).  Therefore,  workload  data  was  collected  and  analyzed  for  a  total  of  41  missions. 

Due  to  the  requirements  of  the  test  board  and  realities  of  field  testing,  it  was  impossible  to  plan  and  achieve 
a  complete  counterbalanced  design.  Therefore,  every  effort  was  made  to  rotate  the  crews  through  the 
various  mission  conditions  without  systematic  bias.  All  crews  participated  at  least  once  in  every  mission 
condition,  except  for  two  crews  who  did  not  operate  Stingray  in  the  manual  mode  for  an  attack  mission. 
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Table  B-2.  ANOVA  table  for  TLX  real-time  workload  data. 


Source 

df 

88 

P 

Between  Subjects  (S) 

Position  (P) 

1 

3428.85 

.65 

P  [crew] 

6 

31805.65 

Within  subjects 

Mission  type  (MT) 

1 

1518.42 

1.64 

MT  x  S 

7 

6486.69 

Mode  (M) 

3 

10178.05 

2.83* 

M  x  S 

21 

25133.81 

Task  (T) 

5 

3867.67 

1.65 

T  x  S 

35 

16427.42 

MT  x  M 

3 

2676.10 

3.20** 

MT  x  M  x  S 

17 

4745.77 

MT  x  T 

5 

429.22 

1.39 

MT  x  T  x  S 

35 

2161.69 

M  x  T 

15 

1027.86 

1.42 

M  X  T  X  S 

103 

4981.53 

MT  X  M  X  T 

15 

818.29 

1.03 

MT  X  M  X  T  X  S 

83 

4376.10 

NOTE:  The  second  factor  for  each  successive  pair  of  factors  is  the  error  term  used  to  determine  the  F-ratio 
for  the  first  factor.  For  example,  Pfcrew]  is  the  between  subjects  error  term  for  the  factor  of  crew  member 
postion  and  MT  x  S  is  the  within  subjects  error  term  for  the  factor  cf  mission  type.  Since  there  are  unequal 
numbers  of  observations  for  different  combinations  of  test  conditions,  a  general  procedure  for  calculating  an 
analysis  of  variance  is  inappropriate.  The  analysis  summarized  in  this  table  was  derived  using  the  general 
linear  model  procedure  given  in  Chapter  20  of  the  SAS  User’s  Guide:  Statistics.  Version  5  (1985,  Cary,  NC: 
SAS  Institute,  Inc).  This  procedure  subtracts 

degrees  of  freedom  from  the  error  term,  where  appropriate  (e.g,  for  Crews  1  and  3  on  Stingray-manual 
attack  missions),  to  compensate  for  missing  data. 

The  Probability  of  an  F-ratio  larger  than  the  one  obtained,  given  the  degrees  shown,  was  greater  than  0.15 
except  where  designated:  '  p= 0.063,  "  p= 0.050. 
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Table  B-3.  Mean  real*time  TLX  workload  ratings  for  target  acquisition  tasks, 


TASK 

MISSION 

TYPE 

BASELINE 

STINGRAY 

AutO 

MODE  OF 

Seni 

OPERATION 

Manual 

Attack 

38 

22 

36 

67 

Search 

Defend 

40 

22 

36 

39 

Attack 

38 

22 

38 

66 

Detect 

Defend 

37 

22 

30 

34 

Attack 

36 

24 

30 

64 

Locate 

Defend 

40 

21 

31 

34 

Attack 

32 

23 

29 

53 

ID 

Defend 

28 

19 

25 

26 

Attack 

30 

25 

32 

52 

Classify 

Defend 

34 

21 

28 

30 

Attack 

28 

24 

30 

48 

Target 

Handoff 

Defend 

31 

22 

29 

33 

NOTE:  Greater  values  indicate  increased  workload  (range  0  -  100). 
Workload  ratings  are  averaged  across  crews  (BCs  and  gunners). 
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Table  B-4.  Mean  real-time  TLX  workload  ratings  by  crew  position. 


MISSION 

TYPE 

CREW 

POSITION 

BASELINE 

STINGRAY 

Auto 

MODE  OF 

Semi 

OPERATION 

Manual 

BC 

29 

20 

31 

56 

Attack 

Gunner 

38 

26 

34 

61 

BC 

27 

22 

29 

30 

Defend 

Gunner 

44 

20 

30 

35 

NOTE:  Greater  values  indicate  increased  workload  (range  0  -  100). 

Workload  ratings  are  averaged  across  the  four  BCs  and  four  gunners  for  all  six  target  acquisition  tasks. 
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Table  B-5.  Mean  real-time  TLX  workload  ratings  for  subscale  dimensions, 


TASK 

node  OF 

OPERATION 

MEN 

SUBSCALE 

PHY  TEM 

DIMENSION 

PER  EFF 

FRU 

SEARCH 

Baseline 

39 

36 

42 

28 

54 

38 

ST-auto 

20 

20 

25 

15 

29 

23 

ST-semi 

39 

27 

39 

34 

46 

39 

ST-manual 

46 

40 

49 

42 

58 

49 

DETECT 

Baseline 

38 

33 

43 

28 

53 

36 

ST-auto 

21 

18 

26 

16 

27 

24 

ST-semi 

34 

29 

31 

30 

43 

40 

ST-manual 

44 

37 

47 

36 

55 

46 

LOCATE 

Baseline 

39 

33 

42 

30 

50 

37 

ST-auto 

21 

19 

26 

19 

30 

21 

ST-semi 

34 

26 

34 

27 

43 

38 

ST-manual 

45 

40 

46 

38 

52 

41 

ID 

Baseline 

32 

27 

33 

24 

37 

30 

ST-auto 

21 

18 

24 

17 

23 

19 

ST-semi 

30 

24 

25 

25 

36 

30 

ST-manual 

39 

32 

37 

27 

45 

32 

CLASSIFY 

Baseline 

35 

28 

33 

26 

43 

30 

ST-auto 

24 

19 

25 

17 

28 

23 

ST-semi 

31 

25 

29 

29 

44 

32 

ST-manual 

40 

37 

40 

32 

45 

37 

TARGET  HAN DOFF 

Baseline 

28 

24 

33 

28 

43 

27 

ST-auto 

22 

18 

28 

20 

26 

22 

ST-semi 

29 

22 

31 

26 

41 

33 

ST-manual 

37 

31 

43 

34 

53 

32 

KEY:  MEN  =  mental  demand,  PHY  =  physical  demand,  TEM  =  temporal  demand,  PER  =  perception  of 
performance  success,  EFF  =  effort  expended,  FRU  =  frustration  level. 

NOTE:  Greater  values  indicate  increased  workload  (range  0  -  100).  Workload  ratings  are  averaged  across 
crews  (BCs  and  gunners)  and  mission  types  (attack  and  defend). 
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Table  B-6.  Prospective,  real-time,  and  retrospective  TLX  workload  ratings  for  each  BC 
and  gunner. 


POSITION 

BASELINE 

STINGRAY  MODE  ' 

OF  OPERATION 

Auto 

Semi 

Manual 

ATTACK  MISSION 

BC1 

65 

(22) 

18 

13  (  8) 

9 

8 

(15) 

10 

35 

(*) 

17 

3 

46 

(42) 

62 

16  (19) 

35 

31 

(48) 

41 

38 

(75) 

62 

5 

(18) 

26 

(10) 

15 

(32) 

36 

(*) 

58 

7 

(30) 

34 

(45) 

32 

(43) 

33 

(36) 

41 

GU2 

34 

(24) 

24 

16  (21) 

10 

23 

(21) 

13 

29 

(*) 

11 

4 

37 

(52) 

34 

17  (17) 

12 

26 

(20) 

23 

70 

(50) 

18 

6 

(30) 

35 

(34) 

33 

(47) 

35 

(*) 

38 

8 

(48) 

62 

(46) 

41 

(49) 

56 

(72) 

50 

DEFEND  MIS8ION 

BC1 

(43) 

15 

(19)  10 

(28) 

9 

(37) 

14 

3 

(28) 

46 

(23)  24 

(40) 

28 

(46) 

46 

5 

32 

(10) 

19 

10 

(17)  14 

63 

(12) 

17 

45 

(14) 

12 

7 

57 

(26) 

25 

36 

(37)  24 

50 

(34) 

27 

63 

(30) 

27 

GU2 

(31) 

26 

(12)  14 

(ID 

14 

(16) 

15 

4 

(56) 

28 

(12)  9 

(22) 

21 

(36) 

30 

6 

51 

(48) 

26 

40 

(50)  29 

57 

(43) 

29 

72 

(35) 

29 

8 

21 

(40) 

51 

11 

(32)  32 

21 

(47) 

43 

28 

(44) 

43 

*  =■  missing  data 

KEY:  P  (RT)  R  ,  where  P  =  prospective  ratings,  (RT)  =  real-time  ratings,  and  R  =  retrospective  ratings. 
NOTE:  The  TLX  workload  ratings  are  averaged  across  six  target  acquisition  tasks.  For  the  prospective 
scales,  half  of  the  crewmembers  rated  attack  and  half  rated  defend  missions  only.  This  was  done  to  reduce 
the  number  of  ratings  elicited,  since  prospective  ratings  were  to  include  both  day  and  night  missions. 
However,  only  daytime  mission  ratings  are  included  in  the  data  presented  in  this  report. 
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Table  B-7.  Mean  prospective  and  retrospective  SWORD  workload  ratings  by  task. 


STINGRAY 

MODE  OF 

OPERATION 

MISSION 

TASK 

TYPE 

BASELINE 

Auto 

Semi 

Manual 

Attack 

P 

.46 

.09 

.12 

.33 

R 

.26 

.10 

.16 

.48 

SEARCH 

Defend 

P 

.43 

.09 

.  10 

.38 

R 

.33 

.12 

.  17 

.38 

Attack 

P 

.42 

.12 

.11 

.35 

R 

.29 

.09 

.17 

.45 

DETECT 

Defend 

P 

.47 

.12 

.  11 

.30 

R 

.22 

.10 

.17 

.51 

Attack 

P 

.42 

.12 

.11 

.35 

R 

.32 

.10 

.16 

.42 

LOCATE 

Defend 

P 

.47 

.12 

.11 

.30 

R 

.31 

.10 

.16 

.43 

Attack 

P 

.42 

.12 

.11 

.35 

R 

.25 

.15 

.21 

.39 

ID 

Defend 

P 

.47 

.12 

.11 

.30 

R 

.28 

.16 

.20 

.36 

Attack 

P 

.33 

.18 

.  15 

.34 

R 

.27 

.15 

.20 

.38 

CLASSIFY 

Defend 

P 

.37 

.  19 

.  14 

.30 

R 

.29 

.  14 

.20 

.37 

TARGET  HANDOFF 

Attack 

R 

.23 

.17 

.20 

.40 

Defend 

R 

.23 

.17 

.20 

.40 

PMCS 

R 

.19 

.27 

.27 

.27 

COMMO 

R 

.12 

.23 

.25 

.40 

LAND  NAVIGATION 

R 

.16 

.25 

.29 

.31 

KEY:  P  =  prospective  SWORD  ratings,  R  =  retrospect ivt  SWORD  ratings.  No  prospective  ratings  were  obtained  for  the  TARGET 
HANDOFF,  PMCS,  COMMO,  AND  LAND  NAVIGATION  tasks. 


NOTE:  Greater  values  indicate  greater  proportion  of  workload  relative  to  other  modes  in  same  row.  Values  are  averaged  across  BCs 
and  gunners.  For  prospective  ratings,  the  tasks  of  LOCATE,  ID,  and  CLASSIFY  were  combined  into  one  category  called,  ACQUIRE 
TARGETS.  The  ratings  for  ACQUIRE  TARGETS  were  allocated  to  these  three  tasks  for  comparison  with  real-time  TLX  ratings. 


Table  B-8.  Mean  prospective,  real-time,  and  retrospective  TLX  workload  ratings  by 
mission  type  and  mode  of  operation. 


MISSION 

TYPE 

RATING 

BASELINE 

STINGRAY 

Auto 

MODE  OF 

Semi 

OPERATION 

Manual 

Attack 

PRO-TLX 

46 

15 

22 

43 

RT-TLX 

34 

23 

33 

58 

RET-TLX 

37 

23 

31 

40 

Defend 

PRO-TLX 

40 

24 

48 

52 

RT-TLX 

35 

21 

30 

33 

RET-TLX 

29 

19 

23 

27 

KEY:  PRO-TLX  =  prospective  TLX  ratings,  RT-TLX  =  real-time  TLX  ratings,  and  RET-TLX  = 
retrospective  TLX  ratings. 

NOTE:  TLX  ratings  are  averaged  across  crews  (BCs  and  gunners)  for  six  target  acquisition  tasks. 
Greater  values  indicate  increased  workload  (range  =  0  -  100). 


Table  B-9.  Mean  prospective  and  retrospective  SWORD  ratings  and  real-time  TLX 
workload  ratings  by  mission  type  and  mode  of  operation. 


MISSION 

TYPE 

RATING 

BASELINE 

8TINGRAY 

Auto 

MODE  OF 

Semi 

OPERATION 

Manual 

Attack 

PRO-SWORD 

.40 

.13 

.13 

.34 

NRT-TLX 

.23 

.  15 

.22 

.40 

RET-SWORD 

.31 

.12 

.17 

.40 

Defend 

PRO-SWORD 

.42 

.13 

.  12 

.33 

NRT-TLX 

.29 

.13 

.25 

.28 

RET-SWORD 

.  33 

.12 

.17 

.38 

KEY:  PRO-SWORD  =  prospective  SWORD  ratings,  NRT-TLX  =  normalized  Real-Time  TLX  ratings, 
RET-SWORD  =  retrospective  SWORD  ratings. 

NOTE:  The  workload  ratings  from  the  TLX  absolute  scale  were  converted  to  normalized  (or  proportional) 
values  for  comparison  with  the  ratings  from  the  SWORD  relative  scale.  Values  in  the  table  should  be 
compared  only  to  other  values  in  the  same  row.  These  values  are  averaged  over  crews  (BCs  and  gunners) 
and  the  target  acquisition  tasks  for  the  four  modes  of  operation  within  each  mission  type.  They  show  the 
proportion  of  workload  experienced  for  each  mode  of  operation  relative  to  the  other  three  modes  of 
operaion. 

For  example,  the  four  PRO-SWORD  values  shown  for  attack  missions  indicate  that  there  is  proportionally 
more  workload  experienced  in  the  baseline  condition  (.40)  than  in  the  Stingray-manual  condition  (.34)  and 
that  there  was  proportionaly  more  workload  experienced  for  these  two  conditions  together  (.74)  than  for  the 
Stingray-automatic  and  Stingray-semiautomatic  conditions  (.13  each). 
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Table  B-10.  Retrospective  SWORD  workload  ratings  and  analyses  for  PMCS, 
communications,  and  land  navigation  tasks . 


TASK:  Preventive  maintenance,  checks,  &  service  (PMCS) 

(F ( 3 , 21)  =  1.27,  E  >  .33) 


A  (.27)  M  (.27)  S  (.27)  B  (.19) 


TASK:  Communications  (Commo) 

(F(3 , 21)  =  5.43,  £  <  -01) 


M  (.40)  S  (.25)  A  (.23)  B  (.12) 


TASK:  Land  Navigation 

(F(3 , 21)  =  8.82,  E  <  .001) 


M  (.31)  S  (.29)  A  (.25)  B  (.16) 


KEY:  For  mode  of  operation,  A  =  Stingray- automatic,  S  =  Stingray-semiautomatic,  M  =  Stingray-manual, 
B  =  Baseline 


NOTE:  Retrospective  SWORD  values  are  averaged  over  BCs  and  gunners.  Means  sharing  the  same 
horizontal  bar  are  not  significantly  different  from  each  other. 
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APPENDIX  C 


VERBATIM  RESPONSES  OF  PARTICIPANTS 
TO  THE  STINGRAY 
END-OF-TEST  QUESTIONNAIRE 


On  the  day  following  the  last  test  mission,  the  four  Stingray 
test  crews,  their  platoon  sergeant,  and  platoon  leader  filled  out 
the  end-of-test  questionnaire.  In  addition,  the  troop  commander 
completed  an  abbreviated  version  of  the  questionnaire  primarily 
covering  tactics,  safety,  and  general  Stingray  issues.  Their 
responses  are  presented  verbatim  in  the  following  appendix. 


The  respondents  have  been  coded  as  follows: 


BC  =  Bradley  commander 
GU  =  gunner 
DR  =  driver 

Crew  1  consisted  of  BC1,  GU2 
Crew  2  consisted  of  BC3,  GU4 
Crew  3  consisted  of  BC5,  GU6 
Crew  4  consisted  of  BC7 ,  GU8 


PLT  LDR  =  platoon  leader 
PLT  SGT  =  platoon  Sergeant 
CO  =  troop  commander 

and  DR31* 
and  DR32 
and  DR33 
and  DR34 


When  it  became  apparent  in  the  test  that  an  experienced  driver 
was  a  necessity  for  effective  operation  of  Stingray  by  the  BC, 
Driver  #31  became  the  permanent  Stingray  driver  for  all  the  crews. 
As  a  result,  83%  of  the  record  test  missions  were  completed  with 
DR3 1 . 
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STINGRAY 

END-OF-TEST  QUESTIONNAIRE 


I.  TASK  PERFORMANCE  and  WORKLOAD 

1.  In  what  ways  has  STINGRAY  changed  how  you  accomplish  your 
tasks?  (For  example,  have  you  dropped  any  less  important 
tasks  in  order  to  accomplish  STINGRAY  tasks?  Have  other  crew 
members  picked  up  some  of  the  tasks  you  normally  do?) 

BC  1:  No.  First  the  STINGRAY  gives  us  something  we  never  had 
(i.e.,  range  to  target)  this  is  especially  helpful  in  killing 
targets.  Knowing  when  to  fire  and  plot  grid  coordinates  to 
targets.  It  also  tells  the  gunner  where  the  next  target  is  in 
azimuth  and  mils. 

BC  3:  BC  has  picked  additional  tasks  to  what  he  already  does. 
While  it  helps  the  gunner  locate  targets. 

BC  5:  yes,  gunner  has  identified  targets  and  engaged  without 
myself. 

BC  7 :  in  the  offense  yes-the  workload  is  harder  because  of 
control  of  the  vehicle-my  driver  and  I  did  not  operate  good 
together  due  to  STINGRAY-I  could  not  guide  him  myself.  Although 
harder  it  is  not  impossible. 

CJ  2:  gunner  must  totally  control  turret  and  weapons  systems, 
and  solely  visual  ID  targets 

GU  4:  I  was  scanning  out  of  my  sector  so  STINGRAY  had  full 
view.  (NOTE:  STINGRAY  search  sector  is  aligned  with  gun  LOS,  so 
gunner  would  traverse  turret  to  orient  STINGRAY  towards  targets 
which  were  threats  to  rest  of  Blue  Force) 

C U  6:  the  task  of  target  hand-off  has  basically  been 
nullified.  The  gunner  makes  the  majority  of  decisions  in  target 
engage  ment . 

(  C  8 :  There  is  not  as  many  target  handoffs  between  gunner/BC, 
because  the  range  and  azimuth  is  given. 

LR  31:  As  main  driver  of  the  STINGRAY,  I  had  a  chance  to  see 
and  hear  four  different  crews  in  action.  On  only  one  crew  did  I 
see  a  extremely  drastic  drop  in  tasks  performed.  They  almost 
eliminated  fire  commands  and  spot  reports  while  operating  STINGRAY. 

DR  33:  no 
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2.  Do  you  think  the  principal  operation  of  STINGRAY  should  be  in 
the  BC's  station  (as  it  was  tested)  or  should  it  be  moved  to 
another  crew  member's  (e.g.,  gunner)?  Why  or  why  not? 

BC  1:  no.  It  should  stay  at  the  BC  position.  This  way  the 
gunner  has  less  work  load  to  worry  about  and  can  concentrate  more 
on  gunning. 

BC  3 :  In  the  BCs  position  because  controls  the  vehicle,  while 
the  gunner  handles  gunning. 

BC  5 :  no 

BC  7:  It  should  stay  the  same,  I  would  rather  want  my  gunner 
concentrating  on  hard  target  engagements  than  STINGRAY  engagements. 

GU  2:  BCs  station 

GU  4:  should  be  moved  to  gunner,  BC  needs  to  navigate  and 
know  where  he  is  all  the  time  to  send  spot  reports.  Gunner  should 
have  control  of  STINGRAY  to  lase  all  targets  and  any  immediate 
danger  distroy  them  on  spot  while  BC  is  sending  up  grid 

GU  6:  I  think  on  the  offensive,  the  gunner  should  have 
counter-measure  capabilities.  The  majority  of  the  time  gunner 
picked  up  targets  first,  also  makes  BC  job  easier. 

GU  8:  It  should  stay  in  BC  station  due  to  that  the  BC  is 
trained  more  than  most  gunners.  But  certain  switches  and 
components  should  be  more  accessible  by  the  gunner. 

DR  31:  It  should  be  in  control  of  the  BC  because  he  is  in 
over-all  control  of  the  vehicle  and  a  weapon  like  this  should  be  in 
the  hands  of  the  BC  not  his  subordinates. 

DR  32:  it  should  be  in  the  BC's  station. 

DR  33:  stay  wear  it  is 

DR  34:  it  should  stay  with  BC  because  gunner  is  constantly 
scanning  &  firing. 
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3.  How  often  did  STINGRAY  pick  up  targets  that  you  had  not 
already  detected  by  other  means  (e.g.,  ISU,  reports  from 
others  in  the  force)? 

a)  Auto  mode 


12  3 

never  !  ! 


4 

i 

i 


5 

i 

i 


6 

i 

i 


7  very 
!  often 


BC  l: 
BC  3: 
BC  5: 
BC  7: 
GU  2: 
GU  4: 
GU  6: 
GU  8: 


3.5 


5.5 

6.5 

6.0 

6.0 

5.5 

6.5 


6.5 


b)  Semi-auto  mode 


12  3 

never  !  !  ! 


4 

l 

I 


5 

i 

i 


6 

i 

i 


BC  1: 
BC  3: 
BC  5: 
BC  7: 
GU  2: 
GU  4: 
GU  6: 
GU  8: 


3.5 


5.5 

4.5 

6.0 

6.0 

4.5 

5.5 

4.5 


7  very 
j  often 


c)  Manual  mode 

12  3 

never  !  !  ! 


4 

i 

i 


5 

i 

i 


6 

i 


7  very 
!  often 


BC  1:* 

BC  3: 

BC  5: 

BC  7:  0 

GU  2: 

GU  4: 

GU  6: 

GU  8: 


2.5 

3.0 


2.5 

2.5 


4.5 


( *BC  1:  on  the  move  2.5) 
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4.  How  useful  was  the  range/azimuth  information  displayed  by 
STINGRAY  to  you  in  engaging  targets  with  BFV  conventional 
weapons? 

1  2  3  4  5  6  7 

no  |  _  j  _  |  _  |  _  |  _  |  _  i  very 

difference  from  useful 

baseline  missions 


BC  1: 

BC  3: 

BC  5 :  2.0 

BC  7: 

GU  2: 

GU  4: 

GU  6: 

GU  8: 


4.5 


6.5 


4.0 


5.5 


4.5 


6.5 

6.5 


5.  what  percentage  of  the  time  is  the  BC  commanding  out  of  the 
hatch? 


a)  on 

the 

attack 

_ % 

b) 

on 

the 

defense 

— 

% 

BC 

1: 

75 

GU 

2: 

80 

BC 

1: 

50 

GU 

2: 

30 

BC 

3: 

90 

GU 

4 : 

100 

BC 

3: 

30 

GU 

4 : 

60 

BC 

5: 

99 

GU 

6: 

80 

BC 

5: 

3 

GU 

6: 

50 

BC 

7: 

99 

GU 

8: 

90 

BC 

7: 

75 

GU 

8: 

75 

what  percentage 

of  the 

time 

are  you 

looking 

through  BFV 

optical 

.  sights? 

a)  on 

the 

attack 

_ % 

b) 

on 

the 

defense 

_ 

% 

BC 

1: 

40 

GU 

2: 

99 

BC 

1: 

70 

GU 

2: 

100 

BC 

3: 

20 

GU 

4 : 

100 

BC 

3: 

80 

GU 

4 : 

100 

BC 

5: 

3 

GU 

6: 

100 

BC 

5: 

75 

GU 

6: 

90 

BC 

7: 

1 

GU 

8: 

80 

BC 

7: 

25 

GU 

8: 

75 

C-6 


7.  Do  you  think  STINGRAY  affected  your  performance 
during  the  missions? 

Give  a  number  on  a  scale  of  **10  to  +  10,  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
+10  =  greatly  improved  my  performance 

(a)  Overall  Performance  of  My  Duty  Position: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+  10 

+8 

+  10 

+2 

+  6 

+  10 

-2 

+  8 

Automatic 

Defend 

+  10 

+7 

+10 

+  10 

+8 

+  10 

+  5 

+9 

Attack 

+8 

+2 

-10 

0 

+4 

+  10 

-2 

+5 

Semiautomatic 

Defend 

+  10 

+  6 

+  10 

+7 

+6 

+  10 

+5 

+9 

Attack 

+2 

-8 

-10 

-10 

+i 

0 

-2 

+  1 

Manual 

Defend 

+6 

-5 

+10 

-5 

+2 

+  10 

+  5 

+9 

COMMENT:  GU6:  On  offense  goggles  irritating,  more  time  in  defense  to  engage. 


(b)  Search  for  Targets: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+  10 

+  2 

+  10 

+  3 

+7 

+  10 

0 

+5 

Automatic 

Defend 

+  10 

+  3 

+  10 

+  10 

+8 

+  10 

0 

+2 

Attack 

+8 

+7 

-10 

0 

+4 

+  10 

0 

+  3 

Semiautomatic 

Defend 

+  10 

+8 

+  10 

+7 

+6 

+  10 

0 

+  2 

Attack 

+  2 

+  1 

-10 

-10 

0 

0 

0 

0 

Manual 

Defend 

+  5 

+  2 

+  10 

-3 

+  2 

+  10 

0 

+2 

COMMENT:  GU4:  Stingray  anual  helps  only  as  a  second  pair  of  eyes. 
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7.  (Con't)  Do  you  think  STINGRAY  affected  your  performance 
during  the  missions? 

Give  a  number  on  a  scale  of  -10  to  +  10,  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
+10  =  greatly  improved  my  performance 


(c)  Detcetion  (Discovery  of  Targets) : 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+  10 

+8 

+  10 

+2 

+8 

+  10 

0 

+  5 

Automatic 

Defend 

+  10 

+8 

+  10 

+8 

+9 

+  10 

+2 

+  1 

Attack 

+  10 

+7 

-10 

0 

+6 

+  10 

0 

0 

Semiautomatic 

Defend 

+  10 

+8 

+  10 

+7 

+7 

+  10 

+  2 

+  1 

Attack 

+3 

+2 

-10 

-10 

+  1 

0 

0 

0 

Manual 

Defend 

+6 

+5 

+  10 

0 

+  1 

+  10 

0 

+  1 

COMMENT:  GU4:  Stingray  manual  helps  only  as  a  second  pair  of  eyes. 
GU6:  Pick  up  targets  quicker  than  Stingray. 


(d)  Location  (Determine  Target  Position) : 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

G2 

G4 

G6 

G7 

Attack 

+10 

0 

+  10 

0 

+5 

+  10 

+2 

+4 

Automatic 

Defend 

+  10 

0 

+  10 

+10 

+8 

+  10 

+  2 

+  1 

Attack 

+  10 

+8 

-10 

0 

+  1 

+  10 

+  2 

+  1 

Semiautomatic 

Defend 

+  10 

+8 

+  10 

+9 

+  6 

+  10 

+2 

+  1 

Attack 

+  2 

+  3 

-10 

-7 

0 

0 

0 

0 

Manual 

Defend 

+5 

+5 

+  10 

0 

+  1 

+  10 

0 

0 

COMMENT:  GU6:  Having  Stingray  give  range  is  big  asset. 
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7.  (Con't)  Do  you  think  STINGRAY  affected  your  performance 
during  the  missions? 

Give  a  number  on  a  scale  of  -10  to  +  10,  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
+10  =  greatly  improved  my  performance 

(e)  Classification  (Prioritize  Targets) : 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+  8 

-3 

+  10 

0 

+2 

0 

0 

0 

Automatic 

Defend 

+9 

-2 

+  10 

0 

+  3 

0 

0 

0 

Attack 

+9 

+ 6 

-10 

0 

+  1 

0 

0 

0 

Semiautomatic 

Defend 

+9 

+  7 

+  10 

0 

+2 

0 

0 

0 

Attack 

+5 

+3 

-10 

0 

0 

0 

0 

0 

Manual 

Defend 

+  6 

+  5 

+10 

0 

0 

0 

0 

0 

COMMENT:  BC7:  No  challenge,  there  were  two  types  of  vehicles:  tank/Bradley  easily  classified. 
GU4:  Stingray  does  not  show  what  type  of  vehicle  it  is. 


(f)  Identification  (Recognize  Target  Type): 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+8 

0 

0 

0 

0 

0 

0 

0 

Automatic 

Defend 

+9 

0 

0 

0 

0 

0 

0 

0 

Attack 

+9 

+7 

0 

0 

0 

0 

0 

0 

Semiautomatic 

Defend 

+  10 

+8 

0 

0 

0 

0 

0 

0 

Attack 

+4 

+  3 

0 

0 

0 

0 

0 

0 

Manual 

Defend 

+6 

+5 

0 

0 

0 

0 

0 

0 

COMMENT:  BC7:  No  challenge,  there  were  two  types  of  vehicles:  tank/Bradley  easily  classified. 
GU2:  Stingray  does  not  identify  target  type. 

GU4:  Stingray  does  not  show  what  type  of  vehicle  it  is. 
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7.  (Con't)  Do  you  think  STINGRAY  affected  your  performance 
during  the  missions? 

Give  a  number  on  a  scale  of  -10  to  +  10/  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
+10  =  greatly  improved  my  performance 

(g)  Target  Handoff: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+  6 

+  3 

0 

0 

+3 

+  10 

+  1 

+5 

Automatic 

Defend 

+8 

+5 

0 

+5 

+4 

+  10 

0 

0 

Attack 

+  6 

+2 

-10 

0 

+2 

+  10 

+  1 

+  3 

Semiautomatic 

Defend 

+8 

+  3 

-10 

+3 

+  3 

+  10 

0 

0 

Attack 

+2 

+  1 

-10 

0 

0 

+  10 

0 

0 

Manual 

Defend 

+  4 

+  3 

-10 

-2 

+  1 

+  10 

0 

0 

COMMENT:  BC7:  Most  of  the  tiniJ  my  gunner  ID  targets  before  Stingray  in  the  offense. 


(h)  Employ  BFV  Weapon  Systems: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

GU2 

GU4 

GU6 

GU7 

Attack 

+  10 

0 

+  10 

0 

0 

+  10 

+2 

+7 

Automatic 

Defend 

+  10 

0 

+  10 

-2 

0 

+10 

+2 

+  10 

Attack 

+9 

+6 

0 

0 

0 

+  10 

+2 

+  4 

Semiautomatic 

Defend 

+  10 

+8 

+  3  0 

-2 

0 

+  10 

+  2 

+  7 

Attack 

+  5 

+  3 

0 

0 

0 

0 

0 

+  1 

Manual 

Defend 

+  5 

+  5 

0 

-2 

0 

0 

0 

+  5 

COMMENT:  BC7:  Because  of  the  low  position  of  the  laser  you  have  to  sit  higher  on  a  berm  for  it  to  scan; 
you’re  exposed  more  because  of  it. 

GU6:  Range  helps  gunner  engage  targets  with  better  first  round  hit  probabilities. 
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7.  (Con't)  Do  you  think  STINGRAY  affected  your  performance 
during  the  missions? 

Give  a  number  on  a  scale  of  -10  to  +  10,  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
-1-10  =  greatly  improved  my  performance 


(i)  COMMO: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

Attack 

+  10 

-3 

-10 

-4 

Automatic 

Defend 

+  10 

-2 

-10 

-1 

Attack 

+8 

-6 

-10 

-4 

Semiautomatic 

Defend 

+9 

-4 

-10 

-1 

Attack 

+3 

-9 

-10 

-4 

Manual 

Defend 

+  5 

-6 

-10 

-1 

COMMENT:  Commo  was  bad.  We  had  to  hold  the  hand  mike  inside  the  CVC. 


(j)  Land  Navigation: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

Attack 

+  10 

0 

0 

0 

Automatic 

Defend 

+  10 

0 

0 

0 

Attack 

+  9 

-5 

-10 

-5 

Semiautomatic 

Defend 

+  10 

0 

0 

0 

Attack 

+  1 

-10 

-10 

-10 

Manual 

Defend 

+8 

0 

0 

0 

Comment:  BC7:  It’s  hard  to  control  the  driver  when  you’re  in  the  turret. 
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7.  (Con't)  Do  you  think  STINGRAY  affected  your  performance 
during  the  missions? 

Give  a  number  on  a  scale  of  -10  to  +  10,  where 
-10  =  greatly  reduced  my  performance 

0  =  made  no  difference  in  my  performance 
+10  =  greatly  improved  my  performance 

(k)  Command  and  Control: 


MODE 

MISSION 

TEST 

PERSONNEL 

BC1 

BC3 

BC5 

BC7 

Attack 

+8 

0 

0 

0 

Automatic 

Defend 

+  10 

0 

0 

+3 

Attack 

+8 

-2 

-10 

0 

Semiautomatic 

Defend 

+8 

-2 

0 

+  3 

Attack 

+1 

-5 

-10 

-1 

Manual 

Defend 

+4 

-3 

0 

0 
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II.  SAFETY 


1.  Does  STINGRAY  pose  any  particular  health  or  safety  hazards  to 
user  personnel? 

CO:  no  I  don't  think  so  but  fratricide  is  a  definite  factor 

to  be  considered 

PL  LDR :  no 

PL  SGT:  no  if  used  properly 
BC  1 :  no 

BC  3:  yes,  the  goggles  is  used  for  safety,  but  during  night 
missions  its  hard  to  navigate  with  goggles 

BC  5 :  — 

BC  7 :  no 

GU  2:  yes,  burn  your  eyeballs 
GU  4 :  no 
GU  6:  no 
GU  8 :  no 

DR  31:  only  accidental  laser  beam  firing 

DR  32:  it  can  cause  blindness  with  carelessness  of  crew. 

DR  3  3 :  no 

DR  34:  only  at  night  because  it  is  extremely  difficult  to  see 
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x 


2.  How  concerned  were  you  about  your  own  health  and  safety 

whenever  you  were  in  close  proximity  to  the  STINGRAY  vehicle? 

a)  during  the  conduct  of  this  test 


not 

1 

2  3  4  5 

6  7 

concerned 

i 

1 

i  i  i  i 

■  i  i  i 

|  i  very 

at  all 

concerned 

CO: 

2.5 

PLT  LDR: 

1.5 

PLT  SGT:* 

3.5 

BC  1: 

2.5 

BC  3: 

5.5 

BC  5: 

1.5 

BC  7: 

4.0 

GU  2: 

1.5 

GU  4: 

2.5 

GU  6: 

2.5 

GU  8: 

5.5 

DR  33: 

1.5 

DR  32: 

6.5 

DR  31: 

2.0 

DR  34:** 

2.5 

*  PLT  SGT :  concerned  enough  not  to  take  off  my  eye  protection 
**  DR  34:  only  at  night 

b)  during  an  actual  battle  with  STINGRAY  at  its  full 
capabilities  (e.g.,  accidents,  fratricide) 
not  1  2  3  4  5  6  7 

concerned  \  _  |  _  j  _  |  _  !  _  !  _  j  very 

at  all  concerned 


CO: 

PLT  LDR: 
PLT  SGT: 
BC  1: 

BC  3: 

BC  5: 

BC  7:* 
GU  2: 

GU  4: 

GU  6: 

GU  8: 

DR  33: 

DR  32: 

DR  31: 

DR  34:* 


1.5 


1.5 


2.5 


3.5 


4.5 


5.5 


6.5 

6.5 


2.5 

2.5 

3.5 


2.5 


4.0 


6.5 


*  BC  7  &  DR  34:  only  at  night 
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3.  Do  you  feel  the  safety  procedures  utilized  during  this  test 
need  any  improvement?  If  so,  then  please  explain. 

CO:  No,  but  a  full  explanation  of  what  the  system  tested  can 

do  (safety  wise)  needs  to  be  better  explained  to  ALL  participants. 
PLT  LDR:  no.  no  injuries  =  effective  safety  program 
PLT  SGT:  — 

BC  1:  no 
BC  3 :  no 
BC  5:  — 

BC  7 :  no  night  time  exercises  due  to  the  goggles,  I  thought 
it  was  very  unsafe 
GU  2 :  no 
GU  4 :  no 
GU  6 :  no 
GU  8 :  no 

DR  31:  safety  procedures  were  very  good  during  this  test  and 
also  they  were  never  any  incidents  of  failure  to  follow  safety 
precaution  due  to  the  fact  of  laser  beam  weapons. 

DR  32:  no 
DR  3  3 :  no 


III.  TRAINING 


1. 


How  would  you  rate  the  training  you  received  on  STINGRAY 
operations? 


1  2  3  4  5  6 

i  i  i  i  i  i 


poor  !!!!!! 

PLT  LDR: 

5.5 

PLT  SGT: 

5.5 

BC  1: 

5.5 

BC  3: 

5.5 

BC  5: 

6.5 

BC  7: 

4.0 

GU  2: 

6.5 

GU  4: 

6.5 

GU  6: 

4.5 

GU  8: 

5.5 

DR  31: 

6.0 

DR  32: 

4.5 

DR  33: 

6.5 

DR  34: 

3.5 

7 

!  excellent 
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2.  How  would  you  rate  the  STINGRAY  operator's  manual? 


l 

Poor  j 


2 

i 

i 


3  4  5  6  7 

J  _  j  _  j  _  j  _  J  Excellent 


PLT  LDR: 

PLT  SGT: 

BC  1: 

BC  3 :  1.5 

BC  5: 

BC  7: 

GU  2: 

GU  4 :  1.5 

GU  6: 

GU  8: 

DR  31: 

DR  32: 

DR  33: 

DR  34: 


4.5 


5.5 


6.5 


4.5 


6.0 

6.5 


4.5 


4.5 


6.5 

7.0 


5.5 

5.5 


3.  How  useful  was  the  condensed  version  of  STINGRAY  operator's 
manual? 


no  help  1  2  3  4  5  6  7  very 

at  all  J  _  |  _  j  _  |  _  J  _  !  _  j  useful 


PLT  LDR: 
PLT  SGT: 
BC  1: 

BC  3: 

BC  5: 

BC  7: 

GU  2: 

GU  4: 

GU  6: 

GU  8: 

DR  31: 

DR  32: 

DR  33: 
DR  34: 


4.5 


3.5 


4.5 


5.5 


4.5 


5.0 

4.5 

4.5 


6.5 


7.0 

6.5 

6.5 


6.5 


6.5 
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4.  Do  you  have  any  comments  or  suggestions  about  how  to  improve 
the  STINGRAY  training  for  future  crews? 


PLT  LDR :  no 

PLT  SGT:  more  time  on  STINGRAY  operations,  hands  on  training 
with  knowledgeable  personnel  prior  to  using 

BC  1 :  no 

BC  3:  yes,  cut  out  the  specific  details  and  get  straight  to 
the  point 

BC  5:  don't  hold  information  back.  Let  them  know  the  system. 
(NOTE:  BC  explained  upon  further  questioning  that  he  wants  to  know 
exactly  how  the  STINGRAY  system  (laser  functioning,  etc)  works,  he 
is  especially  concerned  in  regards  to  the  maintenance  that  would  be 
required  on  the  STINGRAY  system.  He  said  he  watched  the  contractor 
replace  components  without  explanation  to  the  crews  and  BC  wants  to 
know  everything  about  the  system  he  is  operating.) 

BC  7 :  more  hands  on  prior  to  actual  exercises 

GU  2:  yes,  use  the  four  STINGRAY  crews  from  this  test  as 
instructors  for  future  crews  because  we  have  had  hands  on 
experience 

GU  4 :  for  gunner  instead  of  goggle  use  a  protective  lens 
placed  in  the  dust  cover  of  ISU 

GU  6 :  no 

GU  8 :  no 

DR  31:  Power  up/ reset/power  down  procedures  should  be 
practiced  more  to  insure  user  faith  in  using  the  weapon  system. 


DR 

32: 

no 

DR 

33: 

no 
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5.  If  you  were  to  pass  along  any  tips  about  STINGRAY 
operations/functions,  what  would  they  be? 


PLT  LDR:  none.  I  found  the  STINGRAY  very  straightforward. 

PLT  SGT:  — 

BC  1:  Not  to  depend  solely  on  STINGRAY  for  protection.  It  is 
a  very  useful  system,  but  sometimes  crews  start  to  depend  solely  on 
STINGRAY . 

BC  3:  be  flexible 

BC  5:  in  offense  engagements  don't  use  semi  or  manual. 

BC  7:  watch  vehicle  position  (level  vehicle)  in  manual,  if 
your  gunner  picks  up  a  target  hit  stop  scan  and  the  laser  will  line 
up  to  the  gun  then  laze  away! 

GU  2:  utilize  auto-mode  more  often 

GU  4:  If  on  STINGRAY  scan  out  of  your  sector  to  acquire 
targets  for  all  instead  of  just  yourself. 

GU  6:  not  to  rely  on  STINGRAY  too  much.  Often  the  individual 
will  detect  target  first. 

GU  8:  already  answered  before  (NOTE:  check  the  HFEA 
questionnaire) 

DR  31:  learn  to  use  the  STINGRAY  along  with  present 
procedures  instead  of  dropping  procedures  to  use  the  Stingray! 
(NOTE:  when  asked,  driver  said  tasks  dropped  included  things  such 

as  spot  reports,  fire  commands) . 

DR  33:  it  helps  to  detect  targets  well 


C-18 


6.  Do  you  feel  that  personnel  need  any  additional/ spec! fie 

skills  or  aptitudes  to  effectively  operate  STINGRAY  (other 
than  what  is  currently  required  for  your  MOS)? 


PLT  LDR:  no 
PLT  SGT :  — 

BC  1:  no  it  is  an  easy  system  to  operate. 

BC  3 :  no 

BC  5:  know  how  to  deploy  STINGRAY  effectively. 

BC  7:  just  more  patience 
GU  2 :  no 
GU  4 :  no 
GU  6 :  no 

GU  8:  yes,  need  to  learn  about  lasers  and  STINGRAY 
capabilities.  (NOTE:  GU  explained  that  this  info  would  help  to 
understand  how  to  operate  the  system  better,  they  just  don't  want 
to  know  "push  this  button"  level  of  info) 

DR  31:  none  that  I  can  think  of  now 

DR  3  3 :  no 
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IV.  PERFORMANCE 


1.  In  general/  the  survivability  of  the  Blue  Force  was  due  to 


BLUE 

ATTACK 

BLUE 

DEFEND 

% 

% 

presence/ absence 

Of  STINGRAY 

% 

% 

tactics  employed 

% 

% 

personnel 

% 

% 

instrumentation 

(e.g./  MILES) 

% 

% 

other 

NOTE:  these  should  add  up  to  100% 


CO:  20 

50 

PL  LDR:  20 

30  PL  SGT: 

20 

30 

40 

25 

40 

30 

60 

50 

40 

25 

40 

40 

5 

5 

- 

- 

- 

- 

15 

15 

_ 

... 

— 

— 

— 

— 

BC  1:  30 

0 

BC  3:  30  30 

BC  5:  50 

80 

BC  7:  30  35 

20 

50 

30  10 

20  10 

40  40 

40 

40 

10  30 

20 

5 

25  25 

10 

10 

30  30 

10 

5 

0  0 

0 

0 

0  0 

0 

0 

5  5 

GU 

2: 

25 

50 

GU  4:  50  60  GU  6: 

15 

25 

GU 

8: 

50 

75 

25 

10 

5  5 

45 

35 

37 

12 

50 

40 

5  5 

25 

25 

10 

10 

0 

0 

40  30 

15 

15 

2 

2 

0 

0 

0  0 

0 

0 

1 

1 

DR 

34: 

10 

50 

DR  32:  60  40  DR  33: 

10 

50 

DR 

31: 

25 

40 

70 

20 

40  60 

20 

10 

30 

25 

5 

15 

50  50 

30 

10 

15 

10 

15 

15 

50  50 

10 

10 

0 

0 

0 

0 

0  0 

30 

20 

30 

25 

*  GU  8  and  DR  31:  other  =  terrain 
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2.  Row  would  you  tactically  deploy  STINGRAY  on  the  defense  and 
attack?  Please  include  your  reconmendations  for  where  to 
position  it  in  the  formation ,  how  many  STINGRAYS  should  be 
in  a  platoon  or  company,  what  mode  (Auto,  Semi,  Manual)  of 
operation (s)  would  be  best,  should  STINGRAY  be  in  the  Platoon 
Leader's,  sergeant's  or  company  Commander's  vehicle? 

a)  on  the  defense: 

CO:  use  at  the  flanks  at  least  one  per  platoon,  interlocking 

fields  of  fire,  semi  auto  or  auto  mode  best  to  ID  friend  or  foe 

PLT  LDR:  use  in  auto  forward,  drop  back  to  defensive  belt  & 
switch  to  semi-auto  to  avoid  fratricide.  Do  not  put  on  key  leaders 
vehicle.  Use  St  mostly  for  target  acquisition  &  to  enhance  spot 
reporting. 

PLT  SGT:  position  the  STINGRAY  with  the  best  field  of  fire 
near  the  defending  force  but  not  to  close  enough  to  enable  the 
enemy  to  engage  it  with  the  defending  force.  A  dismounted  mode 
would  be  ideal  for  better  concealment.  The  defending  vehicles 
would  be  the  priority  targets  for  the  enemy  not  the  St  vehicle 
which  in  its  current  form  is  easily  identified  at  great  distances. 

BC  1:  first,  two  STINGRAY  to  a  platoon,  put  on  the  section 
leaders  tracks.  Tne  formation  for  defense  would  be: 

STINGRAY  wingman  PLdr  PISgt  wingman  STINGRAY 

with  the  field  of  view  and  regard  the  STINGRAY  can  cover  the  flanks 
plus  crossfire  across  the  center  sector. 

BC  3:  in  the  center  on  semi  with  tanks.  It  would  be  a  squad 
leader  in  the  position. 

BC  5:  system  works  for  himself 

BC  7 :  STINGRAY  should  be  placed  center  sector  to  defend  the 
whole  sector  if  a  force  should  come.  Srong  East  West  North  South 
it  should  dictate  were  STINGRAY  is  places  [sic]. 

GU  2:  STINGRAY  center  sector,  1/platoon,  auto  mode,  Pit  SGT 

GU  4 :  STINGRAY  in  center  sector 

GU  6:  Have  2  per  platoon,  have  one  cover  frontal  area,  other 
covers  a  vulnerable  flank 

GU  8:  Platoon's  Sergeants  position,  one  per  platoon.  And 
maybe  use  it  in  center  sector  to  cover  the  whole  platoon. 

DR  31:  normal  employment  of  STINGRAY  during  defend  missions 

DR  33:  in  the  middle  so  it  could  protect  more  or  less  the 
whole  pit,  troop,  etc  and  not  just  the  left  or  right  side 

DR  32:  STINGRAY  should  be  in  the  middle.  The  Company  should 
have  one  STINGRAY  per  platoon.  The  mode  of  Defense  should  be  auto. 
The  Platoon  Ldr  should  have  the  STINGRAY. 

DR  34:  Stingray  should  be  placed  in  the  rear  of  the  defense 
to  provide  maximum  usage.  There  should  be  one,  maybe  two,  STINGRAY 
in  the  defense  depending  on  how  large  an  area  is  defended  and  how 
many  enemy  are  expected.  Auto  should  be  used  unless  forward 
elements  are  required  to  fall  back. 
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(Con't)  How  would  you  tactically  deploy  STINGRAY  on  the 
defense  and  attack?  Please  include  your  recommendations  for 
where  to  position  it  in  the  formation,  how  many  STINGRAYS 
should  be  operation (s)  would  be  best,  should  STINGRAY  be  in 
the  Platoon  Leader's,  Sergeant's  or  Company  Commander's 
vehicle? 

b)  on  the  attack: 

CO:  must  be  on  auto  mode  or  it  distracts  the  vehicle 
commander,  use  again  on  flanks,  fratricide  could  be  a  problem 

PLT  LDR:  use  STINGRAY  just  ahead  of  tanks  to  give  their 
firepower  maximum  protection  and  target  acquisition  information 
just  before  the  armor  punch.  Then  switch  STINGRAY  to  semi-auto  & 
place  with  base-of-fire  element. 

PLT  SGT:  use  STINGRAY  as  a  base  of  fire  element  either  as  a 
single  vehicle  or  as  a  section.  Positioned  just  off  the  axis  of 
advance  to  provide  better  ability  to  detect  targets.  Another 
element  used  as  a  base  of  fire  element  from  another  direction. 

BC  1:  still  two  per  platoon,  same  two  vehicles.  I  would  use 
the  three  team  concept  (ie.,  the  two  wingman  scouts  on  the  flanks 
to  protect  the  STINGRAY  from  flank  shots) . 

wingman  wingman 

STINGRAY  PISgt  PILdr  STINGRAY 

BC  3:  in  the  center  on  auto  with  a  squad  leader  in  it 

BC  5:  move  with  others  until  contact,  break  contact  get  a 
good  defensive  position  to  lase  targets 

BC  7:  STINGRAY  should  be  places  in  a  high  over  watch.  Then 
while  the  main  force  move  it  can  scan,  then  bound  up  according  to 
terrain  and  attack  etc.  it  should  not  be  forward. 

GU  2:  STINGRAY  in  center  back,  auto  mode,  Pit  SGT 

GU  4:  wedge  formation,  STINGRAY  in  middle  (point  vehicle) 

GU  6:  have  2  per  platoon,  place  one  with  each  scout  section. 
Get  rid  of  semi. 

GU  8:  one  per  platoon,  platoon  sergeants  vehicle  and  use  in 
overwatch  position  center  sector. 

DR  31:  it  should  be  positioned  approx  300-500  meters  behind 
forward  elements  to  allow  its  survivability  but  also  allows 
STINGRAY  to  cover  and  protect  those  forward  elements 

DR  33:  basically  the  same 

DR  32:  The  STINGRAY  should  be  in  the  middle,  in  the  semi 
mode,  on  the  Platoon  Ldrs  vehicle. 

DR  34:  STINGRAY  should  be  kept  in  the  center  of  the  rear 
elements  to  be  able  to  flow  to  the  most  heavily  defended  side.  The 
STINGRAY  should  be  used  in  semi-auto  or  manual  and  have  only  one  in 
the  attack  to  prevent  confusing  when  battle  is  at  close  quarters. 
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.  (Con't)  How  would  you  tactically  deploy  STINGRAY  on  the 

defense  and  attack?  Please  include  your  recommendations  for 
where  to  position  it  in  the  formation,  how  many  STINGRAYS 
should  be  operation (s)  would  be  best,  should  STINGRAY  be  in 
the  Platoon  Leader's,  Sergeant's  or  company  Commander's 
vehicle? 

c)  any  other  comments  about  tactics: 

CO:  — 

PLT  SGT:  — 

PLT  LDR:  Stingray's  major  contribution  is  in  target 
acquisition.  Even  without  a  countermeasure  capability,  STINGRAY 
will  save  very  important  combat  power  simply  by  alerting  commanders 
to  the  presence,  shape,  and  extent  of  a  threat  fire  sack. 

BC  1 :  na 

BC  3 :  — 

BC  5 :  — 

BC  7:  The  PL  Ldr  &  SGT  and  CO  have  enough  to  worry  about,  one 
per  pit  would  work  fine  in  my  opinion 

GU  2 :  no 

GU  4:  — 

GU  6:  — 

GU  8 :  none 

DR  31:  please  do  not  employ  the  STINGRAY  on  a  Platoon  Leader's 
vehicle,  they  have  a  extremely  large  number  of  tasks  and  adding 
STINGRAY  would  "be  the  straw  that  broke  the  camel ' s  back" .  I 
believe  Senior  scout  should  operate  the  STINGRAY. 

DR  33:  should  have  STINGRAY  one  to  a  platoon  on  Pit  Sgt 
track. 

DR  34:  I  noticed  during  the  last  few  missions  that  when 
attacking  several  Blue  elements  reached  the  destination  and  were 
going  in  circles  and  ended  up  shooting  other  blue  elements.  If  the 
STINGRAY  was  on  auto  and  Blue  Forces  have  shutters,  then  half  of 
the  Blue  Forces  would  have  been  blinded.  While  attacking  the 
STINGRAY  should  be  used  in  semi-auto  or  manual  to  prevent  confusion 
and  injury  to  friendly  troops. 
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3.  How  useful  was  the  target  position  information  relayed  by  the 
STINGRAY  crew  to  the  rest  of  the  Blue  Force? 

no  1234567 

difference  j  _ _ \  _  J  _  |  _  j  _  j  _  j  very 

from  baseline  missions  useful 


CO:* 

2.5 

PLT  LDR: 

5.5 

PLT  SGT: 

6.5 

BC  1: 

6.5 

BC  3: 

4.5 

BC  5: 

5.5 

BC  7:* 

o 

• 

VO 

GU  2: 

4.5 

GU  4: 

6.5 

GU  6: 

2.5 

GU  8: 

5.5 

DR  32: 

6.5 

DR  33: 

5.5 

DR  34: 

4.5 

DR  31: 

6.0 

*  BC  7:  range  was  good  for  fire  control 

*  CO:  range  info  good  but  azimuth  info  useless 
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4.  Do  you  think  STINGRAY  accomplished  its  goal  of  delaying  the 
Red  Force's  fire  so  that  the  Blue  Force  could  engage  the 
enemy  with  BFV  conventional  weapons?  Why  or  why  not? 

CO:  yes  in  the  defense — good  defensive  weapon  when  tied  in 

with  other  Stingrays 

PLT  LDR:  can't  tell,  a  good  defender  tries  to  hold  his  fire 
until  you're  well  into  the  fire  sack  anvvay 

PLT  SGT:  yes,  it  provides  time  for  Blue  Force  vehicles  to 
locate  and  lay  their  weapons  while  Red  Forces  cannot  use  their 
optics 

BC  l:  yes,  anytime  we  can  get  a  4  -  5  sec  advantage,  it  is 
very  helpful  our  vehicles  were  able  to  take  on  10  enemy  vehicles 
and  accomplish  the  mission  80%  of  the  time. 

BC  3:  yes,  because  it  had  caused  shutters  to  close  and 
delayed  Red  from  engaging 

BC  5 :  yes 

BC  7 :  In  the  beginning  yes  but  Red  got  smart  and  would  not 
orientate  until  STINGRAY  was  killed  -  overall  yes  it  did  delay  the 
Red  Force  -  it  forced  them  to  change  tactics 


GU  2: 

yes. 

put  out  their  sights 

GU  4: 

yes 

GU  6: 

yes 

GU  8: 
they  could 

yes,  until  the  Red  Force  position  to  the  rear  until 
engage  the  Blue  Force 

DR  33:  yes,  they  couldn't  see  us.  plain  as  that. 

DR  31:  yes,  causing  the  simulated  blinding  of  optics  gives 
Blue  Force  more  time  to  engage  targets.  Enemy  "Red  Force"  gunners 
can  not  kill  what  they  can  not  see. 

DR  32:  At  first  it  did.  Because  at  first  the  Red  Force  must 
have  been  a  little  bit  scared  to  get  blind. 

DR  34:  Yes,  because  Red  Force  turned  their  turrets  away  until 
Blue  was  in  range.  It  worked  defensively  better  than  offensively. 
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5.  What  tactics  do  you  think  the  Red  Force  used  in  response  to 
STINGRAY?  (in  Red  Defend  and  Red  Attack  missions) 

CO:  turned  sights  away  until  enemy  came  close 

PLT  LDR:  Red  Defend-  did  not  point  optics  at  STINGRAY  until 
they  knew  we  were  within  range.  Red  Attack — no  countermeasure  for 
STINGRAY 

PLT  SGT:  maintain  gun  orientation  far  enough  off  the  target 
that  STINGRAY  would  not  be  able  to  detect  targets,  once  the 
commander  identified  targets  he  would  lay  gun  on  target  and  fire. 

BC  1:  it  rendered  their  sights  inoperative  thus  causing  them 
to  hip  shot  with  the  naked  eye  making  it  very  hard  to  get  direct 
hits  on  us. 

BC  3:  Red  attack  will  probably  stay  the  same  -  all  out 
Red  defence  might  differ  with  score  camouflage 

BC  5:  I  don't  think  there  is  a  tactic  than  can  be  used 
effectively. 

GU  2:  orientated  gun  tubes  to  the  rear  until  within  range, 
cheating  by  re-keying  their  MILES 

GU  4:  ? 

GU  6:  A  couple  of  vehicles  didn't  deploy  sights  on  the  axis 
of  advance  on  offense.  On  defense,  always  a  vehicle  that  was  on 
flank,  not  affected  by  STINGRAY. 

GU  8:  They  positions  their  turrets  away  from  the  suspected 
position  Of  STINGRAY. 

DR  31:  they  would  use  blitz  (high  speed)  maneuvers  in  an 
attempt  to  get  a  few  of  their  vehicles  to  the  objective  and 
hopefully  securing  it. 

DR  32:  after  their  shutter  closed  they  went  to  their  back-up 
sight.  (NOTE:  the  backplate  which  supports  the  surrogate  target 
sight  effectively  blocks  usage  of  the  auxiliary  sights  on  the  BFVs, 
so  this  could  not  be  the  case) . 

DR  34:  Defensively  they  turned  their  turrets  away. 
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6.  Do  you  think  STINGRAY  would  be  effective  on  any  other  types 
of  combat  vehicles  (e.g./  tanks,  wheeled  vehicles,  air 
defense  systems)? 

CO:  useful  on  tanks  just  as  it  is  on  a  Bradley,  in  fact  maybe 

more  useful  since  enemy  forces  key  on  tanks  &  tank  killing  rather 
than  scouts 

PLT  LDR:  more  effective  on  tanks  for  its  countermeasure 
function.  More  effective  on  BFVs  for  its  target  detection 
function. 

PLT  SGT:  wheeled  vehicle  for  better  maneuverability  and 
easier  to  conceal  and  camouflage 


BC  1:  yes,  tanks  and  humvees 

BC  3:  HMMV 

BC  5:  air  defense  systems 

BC  7 :  you  can  make  anything  work 

GU  2:  no-STINGRAY  should  go  with  the  first  people  to  go  into 
combat-SCOUTS ! 

GU  4:  yes 

GU  6:  on  light  units  which  deploy  HMMWVs 

GU  8:  yes,  maybe  helicopter,  air  defense,  tank 

DR  31:  a  wheel  vehicle  could  give  it  a  steady  platform  to 
fire  along  with  high  speed  movement  and  stealth. 

DR  32:  maybe 

DR  33:  I  think  it  would  be  good  on  tanks. 

DR  34:  I  believed  STINGRAY  would  be  better  on  a  smaller  more 
maneuverable  vehicle  (hummer) . 
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7.  Are  there  any  effects  of  the  environment  on  the  effective  use 
of  STINGRAY?  (e.g.,  weather,  terrain,  day/night  operations) 

CO:  in  present  configuration  STINGRAY  ineffective  at  night 

PLT  LDR:  Yes.  Terrain,  improperly  used,  can  completely 
destroy  the  STINGRAY  advantage. 

PLT  SGT:  night  operations,  laser  goggles  impede  night  vision 
to  a  point  you  cannot  see  anything 

BC  1:  there  seem  to  be  no  real  effect  on  the  system 

BC  3:  night  time  operations  if  not  effective  enough  with 
protective  goggles 

BC  5:  night  operations 

BC  7 :  due  to  goggles  it  was  dangerous  to  operate  at  night 

GU  2 :  weather-rain 

GU  4 :  no 

GU  6 :  no 

GU  8 :  — 

DR  31:  night  missions  with  the  STINGRAY  with  goggles  is  very 
hard  and  the  success  of  Blue  Force  attacks  were  greatly  reduced  due 
to  slower  speeds,  lack  of  driver's  vision,  and  gunner's  vision. 
(Driver  added  with  further  questioning:  he  couldn't  see  the  dunes 
at  night  with  little  depth  perception,  and  if  STINGRAY  vehicle  took 
a  dune  too  fast  the  bumping  of  the  vehicle  causes  STINGRAY  to  have 
ES  fails) . 

DR  32:  The  terrain  can  be  an  effect  on  STINGRAY. 

DR  33:  none  that  I  noted 

DR  34:  The  night  operation  were  difficult  because  the  night 
sight  is  green  and  so  were  the  goggles. 
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8.  What  do  you  like  best  and  least  about  the  STINGRAY  system? 


CO:  best-excellent  defensive  weapon  &  gives  accurate  range  to 

target  least-limitations  on  the  move  &  in  manual  mode — it  is  a 
distractor  not  combat  multiplier 

PLT  LDR:  Stingray  saves  lives,  provides  excellent  timely 
battlefield  information,  some  countermeasure  protection.  Stingray 
is  too  unreliable. 

PLT  SGT:  Its  ability  to  detect  vehicles  at  great  distances, 
it  picks  up  targets  you  cannot  see  with  just  your  optics. 

BC  1:  liked-auto  mode,  range  display,  early  warning  at  3750m 

BC  3 :  that  it  delayed  Red  Force  and  gave  extra  time  to  Blue 
to  engage 

BC  5 :  — 

BC  7 :  auto  mode  —  manual  mode  is  useless 

GU  2:  ability  to  wash  out  enemy  sights 

GU  4:  best-it  gives  ranges,  least-if  gun  is  scanning  left 
sector  STINGRAY  cannot  scan  right  sector 

GU  6:  gives  you  a  range  and  more  time  to  engage 

GU  8:  once  hit  by  STINGRAY  the  enemy  should  not  be  able  to 
fire  back 

DR  31:  I  loved  the  STINGRAY  in  Auto-mode  on  both  attack  and 
defense  missions  cause  it  allowed  the  BC  to  give  other  vehicles 
general  location  of  enemy  forces.  What  I  liked  least  is  the  large 
smoke  cloud  caused  by  constantly  revving  the  engine.  (NOTE: 

Driver  explained  that  he  had  to  rev  the  engine  to  keep  the  RPMs  up 
to  keep  STINGRAY  powered) . 

DR  32:  Sometimes  it  detects  targets  pretty  fast.  Least  it 
breaks  down  to  much. 

DR  33:  I  found  that  taching  the  engine  out  is  the  least. 

Being  able  to  tell  wear  target  exactly  is  (best) . 

DR  34:  I  don't  like  the  goggles.  (Driver  added  when 
questioned-due  to  worsening  ability  to  see  rather  than  discomfort 
of  wear,  Dr  said  hatch  is  closed  all  the  time  &  he  is  looking 
through  vision  blocks,  also  night  sight  is  green) . 
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9.  What  changes  would  you  recouend  for  future  development? 

CO:  stab  mode  &  operations  at  night  (NOTE:  stab  refers  to 

capability  to  stabilize  Stingray  as  can  be  done  for  50mm  gun) 

PLT  LDR:  Focus  more  on  the  tactical  intelligence  functions  of 
the  Stingray.  The  countermeasure  is  not  so  important. 

PLT  SGT:  — 

BC  1:  move  the  VDU.  possibly  a  HUDS  in  the  CVC 
BC  3:  more  room  needed  in  the  turret 
BC  5:  — 

BC  7 :  no  manual  mode 
GU  2 :  none 

GU  4:  have  STINGRAY  work  independently  (NOTE:  of  gun  LOS) 

GU  6:  — 

GU  8 :  — 

DR  31:  improved  power  source  capabilities 

DR  33:  exhaust 

DR  34:  put  on  other  vehicle 


10.  How  effective  do  you  think  STINGRAY  would  be  in  an  actual 

battle?  (with  its  full  capabilities  and  intended  functions) 

CO:  very  effective  if  by  the  time  it  was  fielded  the  enemy 

didn't  have  a  countermeasure  for  it 
PLT  LDR:  very  effective 
PLT  SGT:  very  effective 

BC  1:  very  effective 

BC  3:  depending  on  time  when  it  is  used,  yes 
BC  5:  excellent 
BC  7 :  — 

GU  2:  100%  more  effective  in  ID  and  killing  targets 

GU  4:  100% 

GU  6:  improve  survivability  by  substantial  margin  on  defense 
GU  8:  If  the  enemy  is  not  aware  of  STINGRAY.  It  could  be 
very  devastating  to  the  enemy. 

DR  31:  I  think  the  effectiveness  of  the  STINGRAY  in  actual 
battle  would  be  a  big  combat  multiplier  and  would  increase 
survivability  of  more  personnel. 

DR  32:  It  depends  on  the  capabilities  and  functions  of  the 

crew. 

DR  33:  unstoppable 

DR  34:  Defensively  it  would  be  great  but  offensively  is 
questionable.  It  appears  to  be  a  little  fragile  and  would  be 
destroyed  easily. 
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11.  Do  you  think  the  Army  should  buy  the  STINGRAY  system? 

CO:  Further  tests  need  to  be  done  &  a  consideration  of  the 

test  should  be  how  hard  is  is  to  countermeasure.  Would  we  end  up 
spending  hundreds  of  thousands  of  dollars  for  a  system  than  can  be 
defeated  with  29  cents  worth  of  window  screen? 

PLT  LDR:  Only  after  we  enhance  its  information  providing 
capabilities  (give  better  info  on  where  enemy  is.  may  a 
triangulation  feature)  &  making  it  more  reliable. 

PLT  SGT :  yes 

BC  1:  yes 
BC  3 :  yes 

BC  5:  if  changes  suggested  are  made 

BC  7 :  yes 

GU  2 :  yes 

GU  4 :  yes 

GU  6:  yes 

GU  8:  yes,  but  on  a  limited  bases  until  more  proven  by  other 
units  with  other  missions. 

DR  33:  If  they  get  it  to  work  properly  all  the  time,  yes. 

DR  31:  Yes,  once  the  little  '’bugs"  have  been  worked  out. 

DR  34:  Yes,  with  a  few  modifications. 


ANY  OTHER  GENERAL  COMMENTS  OR  SUGGESTIONS: 

BC  1:  The  STINGRAY  is  a  very  useful  weapon.  Because  I  think 
not  only  killing  enemy  optics,  it  will  play  a  big  psychological 
warfare  also. 
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